[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:22 dist-test-slave-1xrh krb5kdc[14548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:22 dist-test-slave-1xrh krb5kdc[14548](info): set up 2 sockets
Aug 06 06:41:22 dist-test-slave-1xrh krb5kdc[14548](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 06:41:23.817677 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:23 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:23.937369 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38127
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:24.022387 14564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:24.022604 14564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:24.022639 14564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:24.023813 14564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:24.023890 14564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:24.023917 14564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:24.023945 14564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:24.023970 14564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:24.025578 14564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:24.025988 14564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:24.026278 14564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:41:24.061113 14564 server_base.cc:1034] running on GCE node
W20260806 06:41:24.061295 14570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.061336 14571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.061606 14573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:24.061949 14564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:24.062345 14564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:24.063501 14564 hybrid_clock.cc:648] HybridClock initialized: now 1785998484063476 us; error 73 us; skew 500 ppm
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:24.070649 14564 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:24.071398 14564 webserver.cc:460] Webserver started at http://127.14.51.126:37481/ using document root <none> and password file <none>
I20260806 06:41:24.071722 14564 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:24.071775 14564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:24.072029 14564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:24.073405 14564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "dacba5f7a2534cdc89bea3c25cd7df77"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "67878c83677e884454d3fcf3a9b73253"
server_key_iv: "0aa0007c0fa79cb3514de542e2f2b7e5"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.074059 14564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "dacba5f7a2534cdc89bea3c25cd7df77"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "67878c83677e884454d3fcf3a9b73253"
server_key_iv: "0aa0007c0fa79cb3514de542e2f2b7e5"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.075901 14564 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 06:41:24.076848 14580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.077109 14564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:24.077205 14564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "dacba5f7a2534cdc89bea3c25cd7df77"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "67878c83677e884454d3fcf3a9b73253"
server_key_iv: "0aa0007c0fa79cb3514de542e2f2b7e5"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.077306 14564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:24.088922 14564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:24.091013 14564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:24.091279 14564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:24.096145 14564 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38127
I20260806 06:41:24.096253 14632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38127 every 8 connection(s)
I20260806 06:41:24.096762 14564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:24.098009 14633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:24.101030 14633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: Bootstrap starting.
I20260806 06:41:24.101869 14633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:24.102233 14633 log.cc:824] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:24.103034 14633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: No bootstrap required, opened a new log
I20260806 06:41:24.103487 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 14564
I20260806 06:41:24.103575 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:24.104239 14541 external_mini_cluster.cc:1428] Setting key 4dada6a94d54a26e7ef9d6d9839d1879
I20260806 06:41:24.104987 14633 raft_consensus.cc:348] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } }
I20260806 06:41:24.105242 14633 raft_consensus.cc:374] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:24.105348 14633 raft_consensus.cc:729] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacba5f7a2534cdc89bea3c25cd7df77, State: Initialized, Role: FOLLOWER
I20260806 06:41:24.105504 14633 consensus_queue.cc:260] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } }
I20260806 06:41:24.105652 14633 raft_consensus.cc:388] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:24.105708 14633 raft_consensus.cc:482] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:24.105764 14633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:24.106828 14633 raft_consensus.cc:504] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } }
I20260806 06:41:24.107028 14633 leader_election.cc:302] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dacba5f7a2534cdc89bea3c25cd7df77; no voters:
I20260806 06:41:24.107280 14633 leader_election.cc:288] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:24.107304 14639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:24.107682 14633 sys_catalog.cc:564] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:24.109650 14639 raft_consensus.cc:686] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [term 1 LEADER]: Becoming Leader. State: Replica: dacba5f7a2534cdc89bea3c25cd7df77, State: Running, Role: LEADER
I20260806 06:41:24.109799 14639 consensus_queue.cc:237] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } }
I20260806 06:41:24.110275 14651 sys_catalog.cc:455] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dacba5f7a2534cdc89bea3c25cd7df77. Latest consensus state: current_term: 1 leader_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } } }
I20260806 06:41:24.110388 14651 sys_catalog.cc:458] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:24.110471 14640 sys_catalog.cc:455] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dacba5f7a2534cdc89bea3c25cd7df77" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38127 } } }
I20260806 06:41:24.110581 14640 sys_catalog.cc:458] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77 [sys.catalog]: This master's current role is: LEADER
W20260806 06:41:24.111794 14654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:41:24.111862 14654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:41:24.112075 14653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:24.112360 14653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:24.114399 14653 catalog_manager.cc:1347] Generated new cluster ID: 70ac2cbe849a4d6e967915c283657650
I20260806 06:41:24.114473 14653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:24.124742 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.105532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.14.51.126:38127)
0806 06:41:24.105956 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.105967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.106123 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:41:24.108394 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.108403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.108521 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.108691 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.110872 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.112907 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.114605 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.115493 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.115644 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.119861 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.119865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.119866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.119877 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.122498 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.122914 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.122917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.122918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.123031 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.123539 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.123540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.123540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.123869 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.123999 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.124254 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.124435 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
I20260806 06:41:24.127802 14653 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:24.128382 14653 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:24.133244 14653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dacba5f7a2534cdc89bea3c25cd7df77: Generated new TSK 0
I20260806 06:41:24.133404 14653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:24.162788 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:24.266597 14660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:24.266793 14660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:24.266814 14660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:24.267994 14660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:24.268041 14660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:24.268069 14660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:24.269501 14660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:24.269814 14660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:24.270053 14660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:24.275355 14668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.275285 14665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.275314 14666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:24.275517 14660 server_base.cc:1034] running on GCE node
I20260806 06:41:24.275724 14660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:24.275907 14660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:24.277043 14660 hybrid_clock.cc:648] HybridClock initialized: now 1785998484277026 us; error 32 us; skew 500 ppm
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:24.280169 14660 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:24.280577 14660 webserver.cc:460] Webserver started at http://127.14.51.65:37109/ using document root <none> and password file <none>
I20260806 06:41:24.280821 14660 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:24.280865 14660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:24.281036 14660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:24.282300 14660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "8d6c9620a7f042ae96d149a472962f53"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "f64e02192aa484fcb0e8e4dddc1d3b37"
server_key_iv: "f738a655e10caf54bf74804f5e4492a6"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.282711 14660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "8d6c9620a7f042ae96d149a472962f53"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "f64e02192aa484fcb0e8e4dddc1d3b37"
server_key_iv: "f738a655e10caf54bf74804f5e4492a6"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.284221 14660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:24.284912 14675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.285125 14660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:24.285223 14660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "8d6c9620a7f042ae96d149a472962f53"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "f64e02192aa484fcb0e8e4dddc1d3b37"
server_key_iv: "f738a655e10caf54bf74804f5e4492a6"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.285277 14660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:24.318048 14660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:24.320134 14660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:24.320248 14660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:24.320485 14660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:24.320917 14660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:24.320991 14660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.321017 14660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:24.321033 14660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.328492 14660 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44623
I20260806 06:41:24.328527 14795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44623 every 8 connection(s)
I20260806 06:41:24.328933 14660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:24.338066 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 14660
I20260806 06:41:24.338173 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:24.338325 14541 external_mini_cluster.cc:1428] Setting key dc642833008eaed69ac2cef7f637111d
I20260806 06:41:24.339094 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.329910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:50517 (local address 127.14.51.126:38127)
0806 06:41:24.329967 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.329970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.330164 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:41:24.330682 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.330683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.330698 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.330748 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.331949 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.332441 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.333524 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.333736 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.333822 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.335992 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.335993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.335993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.336001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.337429 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.337883 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.337884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.337884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.337944 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.338254 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.338255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.338255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.338365 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.338442 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.338718 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.338827 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:41:24.339108 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.329867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38127 (local address 127.14.51.65:50517)
0806 06:41:24.330032 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:24.330080 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:24.330574 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:24.330819 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:24.330822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:24.331626 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:24.331841 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.331846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.332593 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.332596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.333403 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.333406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.333833 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.333835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.333949 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.334443 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:24.334451 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:24.335896 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:24.337532 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.337539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.337541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.337789 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.338017 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.338018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.338018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.338167 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.338433 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:24.338440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:24.338596 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 06:41:24.338701 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:41:24.338872 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260806 06:41:24.339572 14796 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38127
I20260806 06:41:24.339658 14796 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:24.339830 14796 heartbeater.cc:511] Master 127.14.51.126:38127 requested a full tablet report, sending...
I20260806 06:41:24.341114 14597 ts_manager.cc:194] Registered new tserver with Master: 8d6c9620a7f042ae96d149a472962f53 (127.14.51.65:44623)
I20260806 06:41:24.342196 14597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:50517
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:24.370918 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:24.456255 14801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:24.456434 14801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:24.456457 14801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:24.457649 14801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:24.457692 14801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:24.457719 14801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:24.459200 14801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:24.459443 14801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:24.459681 14801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:24.465148 14807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.465230 14808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.465149 14811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:24.465562 14801 server_base.cc:1034] running on GCE node
I20260806 06:41:24.465750 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:24.466002 14801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:24.467198 14801 hybrid_clock.cc:648] HybridClock initialized: now 1785998484467179 us; error 31 us; skew 500 ppm
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:24.469770 14801 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:24.470245 14801 webserver.cc:460] Webserver started at http://127.14.51.66:41827/ using document root <none> and password file <none>
I20260806 06:41:24.470492 14801 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:24.470561 14801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:24.470698 14801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:24.471782 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "abf66c1c5e31409cbf19b64a0b3ace85"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "4d7cd3e69b096a96530f7b12ebf35be8"
server_key_iv: "6e01df0fbde53cefc8041b7d6067c4d2"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.472182 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "abf66c1c5e31409cbf19b64a0b3ace85"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "4d7cd3e69b096a96530f7b12ebf35be8"
server_key_iv: "6e01df0fbde53cefc8041b7d6067c4d2"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.473618 14801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:24.474349 14818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.474515 14801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:24.474619 14801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "abf66c1c5e31409cbf19b64a0b3ace85"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "4d7cd3e69b096a96530f7b12ebf35be8"
server_key_iv: "6e01df0fbde53cefc8041b7d6067c4d2"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.474695 14801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:24.484131 14801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:24.485824 14801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:24.485978 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:24.486316 14801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:24.486702 14801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:24.486768 14801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.486811 14801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:24.486840 14801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.496781 14801 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:44905
I20260806 06:41:24.496819 14937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:44905 every 8 connection(s)
I20260806 06:41:24.497164 14801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:24.497290 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 14801
I20260806 06:41:24.497437 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:24.497663 14541 external_mini_cluster.cc:1428] Setting key 6756f9ccb12340bc79255138c1d971c2
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:41:24.507368 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.497958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44437 (local address 127.14.51.126:38127)
0806 06:41:24.498011 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.498013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.498271 (+ 258us) server_negotiation.cc:407] Connection header received
0806 06:41:24.498817 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.498818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.498830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.498941 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.499888 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.500672 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.501963 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.502297 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.502373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.504457 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.504459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.504459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.504467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.506075 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.506433 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.506434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.506435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.506495 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.506728 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.506730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.506730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.506864 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.506934 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.507177 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.507261 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:41:24.507522 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.497942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38127 (local address 127.14.51.66:44437)
0806 06:41:24.498151 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:24.498183 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:24.498716 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:24.498990 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:24.498993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:24.499606 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:24.499785 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.499789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.500763 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.500765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.501878 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.501880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.502379 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.502380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.502476 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.502972 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:24.502979 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:24.504366 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:24.506161 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.506166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.506167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.506361 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.506577 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.506578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.506578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.506663 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.506928 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:24.506931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:24.507027 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 06:41:24.507101 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:41:24.507321 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":138,"threads_started":1}
I20260806 06:41:24.507977 14938 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38127
I20260806 06:41:24.508064 14938 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:24.508265 14938 heartbeater.cc:511] Master 127.14.51.126:38127 requested a full tablet report, sending...
I20260806 06:41:24.508636 14597 ts_manager.cc:194] Registered new tserver with Master: abf66c1c5e31409cbf19b64a0b3ace85 (127.14.51.66:44905)
I20260806 06:41:24.509006 14597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:44437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:24.524036 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:24.605008 14944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:24.605197 14944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:24.605219 14944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:24.606268 14944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:24.606303 14944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:24.606331 14944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:24.607924 14944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:24.608093 14944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:24.608297 14944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:24.613495 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.613495 14952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:24.613487 14949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:24.613682 14944 server_base.cc:1034] running on GCE node
I20260806 06:41:24.614291 14944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:24.614639 14944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:24.615774 14944 hybrid_clock.cc:648] HybridClock initialized: now 1785998484615760 us; error 37 us; skew 500 ppm
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:24.618121 14944 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:24.618530 14944 webserver.cc:460] Webserver started at http://127.14.51.67:45399/ using document root <none> and password file <none>
I20260806 06:41:24.618754 14944 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:24.618853 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:24.619004 14944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:24.620033 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "c8cc33607006421eb2ae8ded47d2bf94"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "aad78c46b46e6febd4d1ea32e865b77a"
server_key_iv: "162c779398625856b62c2cf86c00b26b"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.620386 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "c8cc33607006421eb2ae8ded47d2bf94"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "aad78c46b46e6febd4d1ea32e865b77a"
server_key_iv: "162c779398625856b62c2cf86c00b26b"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.621675 14944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:24.622342 14959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.622514 14944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:24.622617 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "c8cc33607006421eb2ae8ded47d2bf94"
format_stamp: "Formatted at 2026-08-06 06:41:24 on dist-test-slave-1xrh"
server_key: "aad78c46b46e6febd4d1ea32e865b77a"
server_key_iv: "162c779398625856b62c2cf86c00b26b"
server_key_version: "encryptionkey@0"
I20260806 06:41:24.622697 14944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:24.633841 14944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:24.635604 14944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:24.635751 14944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:24.635959 14944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:24.636238 14944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:24.636291 14944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.636330 14944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:24.636358 14944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:24.645273 14944 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:44651
I20260806 06:41:24.645318 15077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:44651 every 8 connection(s)
I20260806 06:41:24.645635 14944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:24.649124 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 14944
I20260806 06:41:24.649209 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:24.649356 14541 external_mini_cluster.cc:1428] Setting key 80fda66c9e4445c1fefbc018c24f9d50
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:24.655066 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.646386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:39653 (local address 127.14.51.126:38127)
0806 06:41:24.646473 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.646475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.646663 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:41:24.647241 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.647242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.647252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.647303 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.648249 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.648732 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.650127 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.650338 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.650415 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.652545 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.652546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.652547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.652554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.653711 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.654096 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.654097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.654098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.654158 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.654434 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.654435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.654436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.654551 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.654623 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.654886 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.654977 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 06:41:24.655241 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.646360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38127 (local address 127.14.51.67:39653)
0806 06:41:24.646549 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:24.646581 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:24.647069 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:24.647352 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:24.647355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:24.647974 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:24.648171 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.648175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.648806 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.648807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.650049 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:24.650051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.650419 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.650420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:24.650559 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.650958 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:24.650965 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:24.652443 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:24.653801 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.653806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.653808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.654021 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.654235 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:24.654236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:24.654236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:24.654365 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:24.654625 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:24.654632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:24.654754 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 06:41:24.654850 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:41:24.655001 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260806 06:41:24.655743 15078 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38127
I20260806 06:41:24.655812 15078 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:24.655963 15078 heartbeater.cc:511] Master 127.14.51.126:38127 requested a full tablet report, sending...
I20260806 06:41:24.656291 14597 ts_manager.cc:194] Registered new tserver with Master: c8cc33607006421eb2ae8ded47d2bf94 (127.14.51.67:44651)
I20260806 06:41:24.656672 14597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:39653
I20260806 06:41:24.661185 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:24.685444 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.678048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.14.51.65:44623)
0806 06:41:24.678212 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.678216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.678225 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:24.678249 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.678251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.678264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.678329 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.678867 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.679635 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.680353 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.680641 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.680693 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.682798 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.682805 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.682805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.682811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.684475 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.684763 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.684766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.684766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.684819 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.684978 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.684979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.684979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.685091 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.685197 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.685246 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.685309 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
W20260806 06:41:24.685875 14704 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42946
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[14548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:24.710179 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:24.703255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.14.51.65:44623)
0806 06:41:24.703310 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:24.703312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:24.703371 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:41:24.703413 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:24.703413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:24.703423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:24.703480 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:24.704008 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.704502 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.705179 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:24.705382 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:24.705431 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:24.707525 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:24.707527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:24.707527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:24.707534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:24.709137 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.709470 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.709471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.709471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.709523 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:24.709712 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:24.709713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:24.709713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:24.709805 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:24.709915 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:24.710006 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:41:24.710080 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:41:24.710644 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 14660
I20260806 06:41:24.716861 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 14801
I20260806 06:41:24.724287 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 14944
I20260806 06:41:24.731345 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 14564
2026-08-06T06:41:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2040 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[15092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[15092](info): set up 2 sockets
Aug 06 06:41:24 dist-test-slave-1xrh krb5kdc[15092](info): commencing operation
krb5kdc: starting...
W20260806 06:41:25.785864 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:25 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:25.902159 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40277
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:40277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:25.984004 15105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:25.984266 15105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:25.984310 15105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:25.985555 15105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:25.985633 15105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:25.985661 15105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:25.985693 15105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:25.985733 15105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:25.987381 15105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:40277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:25.987694 15105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:25.987953 15105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:25.993022 15114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:25.993148 15112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:25.993319 15111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:25.993309 15105 server_base.cc:1034] running on GCE node
I20260806 06:41:25.993582 15105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:25.993815 15105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:25.994953 15105 hybrid_clock.cc:648] HybridClock initialized: now 1785998485994938 us; error 27 us; skew 500 ppm
Aug 06 06:41:25 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998485, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:25.997426 15105 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:25.997802 15105 webserver.cc:460] Webserver started at http://127.14.51.126:37919/ using document root <none> and password file <none>
I20260806 06:41:25.997998 15105 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:25.998040 15105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:25.998126 15105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:25.999089 15105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "35b14205738b4ac681bd21916862b19d"
format_stamp: "Formatted at 2026-08-06 06:41:25 on dist-test-slave-1xrh"
server_key: "d12989f93ccfb1166839f72324d0a5db"
server_key_iv: "43e8545534482fd43ec3f7426880a562"
server_key_version: "encryptionkey@0"
I20260806 06:41:25.999485 15105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "35b14205738b4ac681bd21916862b19d"
format_stamp: "Formatted at 2026-08-06 06:41:25 on dist-test-slave-1xrh"
server_key: "d12989f93ccfb1166839f72324d0a5db"
server_key_iv: "43e8545534482fd43ec3f7426880a562"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.000707 15105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:26.001355 15122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.001506 15105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:26.001605 15105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "35b14205738b4ac681bd21916862b19d"
format_stamp: "Formatted at 2026-08-06 06:41:25 on dist-test-slave-1xrh"
server_key: "d12989f93ccfb1166839f72324d0a5db"
server_key_iv: "43e8545534482fd43ec3f7426880a562"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.001660 15105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:26.031103 15105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:26.032948 15105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:26.033059 15105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:26.036935 15105 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:40277
I20260806 06:41:26.037024 15175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:40277 every 8 connection(s)
I20260806 06:41:26.037361 15105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:26.038266 15176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:26.040500 15176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d: Bootstrap starting.
I20260806 06:41:26.041072 15176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:26.041355 15176 log.cc:824] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:26.042054 15176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d: No bootstrap required, opened a new log
I20260806 06:41:26.043707 15176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } }
I20260806 06:41:26.043851 15176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:26.043874 15176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b14205738b4ac681bd21916862b19d, State: Initialized, Role: FOLLOWER
I20260806 06:41:26.044051 15176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } }
I20260806 06:41:26.044113 15176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:26.044152 15176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:26.044202 15176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:26.044693 15176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } }
I20260806 06:41:26.044801 15176 leader_election.cc:302] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35b14205738b4ac681bd21916862b19d; no voters:
I20260806 06:41:26.044987 15176 leader_election.cc:288] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:26.045024 15179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:26.045204 15179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [term 1 LEADER]: Becoming Leader. State: Replica: 35b14205738b4ac681bd21916862b19d, State: Running, Role: LEADER
I20260806 06:41:26.045241 15176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:26.045295 15179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } }
I20260806 06:41:26.045764 15180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35b14205738b4ac681bd21916862b19d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } } }
I20260806 06:41:26.045855 15180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:26.045993 15181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35b14205738b4ac681bd21916862b19d. Latest consensus state: current_term: 1 leader_uuid: "35b14205738b4ac681bd21916862b19d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b14205738b4ac681bd21916862b19d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40277 } } }
I20260806 06:41:26.046120 15181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:26.046401 15188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:26.046629 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15105
I20260806 06:41:26.046718 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:26.046887 14541 external_mini_cluster.cc:1428] Setting key fb03a3d316e59b3c4213dd090efa8ff1
I20260806 06:41:26.046964 15188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:26.048470 15188 catalog_manager.cc:1347] Generated new cluster ID: c07594a307234c9aba573abfa53ac2c9
I20260806 06:41:26.048524 15188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:26.056779 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.047377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.14.51.126:40277)
0806 06:41:26.047832 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.047840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.047876 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:41:26.047986 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.047992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.048058 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.048185 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.048893 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.049766 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.050529 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.050951 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.051085 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.053480 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.053483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.053484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.053495 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.055205 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.055570 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.055573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.055573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.055672 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.055887 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.055888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.055889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.056127 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.056289 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.056381 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.056621 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":80,"threads_started":1}
I20260806 06:41:26.084270 15188 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:26.084807 15188 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:26.097316 15188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35b14205738b4ac681bd21916862b19d: Generated new TSK 0
I20260806 06:41:26.097517 15188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:26.129964 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:26.224045 15204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:26.224228 15204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:26.224251 15204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:26.225378 15204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:26.225503 15204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:26.225560 15204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:26.227105 15204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:26.227444 15204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:26.227692 15204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:26.232352 15210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:26.232374 15209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:26.232419 15212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:26.232676 15204 server_base.cc:1034] running on GCE node
I20260806 06:41:26.232877 15204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:26.233105 15204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:26.234273 15204 hybrid_clock.cc:648] HybridClock initialized: now 1785998486234278 us; error 45 us; skew 500 ppm
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:26.236838 15204 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:26.237273 15204 webserver.cc:460] Webserver started at http://127.14.51.65:35073/ using document root <none> and password file <none>
I20260806 06:41:26.237498 15204 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:26.237565 15204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:26.237712 15204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:26.238662 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "59f084e004740c00ce65861259780032"
server_key_iv: "5ba975f8689c6d53cc49f45a1e37e55b"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.239006 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "59f084e004740c00ce65861259780032"
server_key_iv: "5ba975f8689c6d53cc49f45a1e37e55b"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.240284 15204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:26.241017 15220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.241184 15204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:26.241277 15204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "59f084e004740c00ce65861259780032"
server_key_iv: "5ba975f8689c6d53cc49f45a1e37e55b"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.241354 15204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:26.250319 15204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:26.252259 15204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:26.252410 15204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:26.252615 15204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:26.252890 15204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:26.252943 15204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.252976 15204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:26.253003 15204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.258858 15204 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:46533
I20260806 06:41:26.258906 15337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:46533 every 8 connection(s)
I20260806 06:41:26.259250 15204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:26.265035 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15204
I20260806 06:41:26.265126 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:26.265290 14541 external_mini_cluster.cc:1428] Setting key 73daaeca2e5e262ae44fac3873522a18
I20260806 06:41:26.268625 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.259915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:50963 (local address 127.14.51.126:40277)
0806 06:41:26.259970 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.259972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.260163 (+ 191us) server_negotiation.cc:407] Connection header received
0806 06:41:26.261189 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.261190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.261204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.261265 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.261792 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.262309 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.263281 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.263476 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.263529 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.265568 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.265570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.265570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.265577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.267280 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.267643 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.267644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.267645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.267711 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.267999 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.268000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.268000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.268097 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.268146 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.268498 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.268563 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:41:26.268893 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.259876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40277 (local address 127.14.51.65:50963)
0806 06:41:26.260033 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.260069 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.261095 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.261315 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.261318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.261549 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:26.261716 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.261720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.262388 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.262388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.263208 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.263210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.263546 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.263547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.263636 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.264053 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:26.264060 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:26.265470 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:26.267353 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.267358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.267360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.267567 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.267798 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.267799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.267799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.267932 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.268160 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:26.268166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:26.268334 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.268477 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.268635 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260806 06:41:26.269333 15338 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40277
I20260806 06:41:26.269423 15338 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:26.269613 15338 heartbeater.cc:511] Master 127.14.51.126:40277 requested a full tablet report, sending...
I20260806 06:41:26.270147 15139 ts_manager.cc:194] Registered new tserver with Master: d14e184b5f4c4ee0ba7b2a5a9132947c (127.14.51.65:46533)
I20260806 06:41:26.270750 15139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:50963
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:26.290367 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:26.370637 15344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:26.370827 15344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:26.370851 15344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:26.372000 15344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:26.372042 15344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:26.372069 15344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:26.373432 15344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:26.373607 15344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:26.373767 15344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:26.378686 15350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:26.378692 15352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:26.378711 15349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:26.379330 15344 server_base.cc:1034] running on GCE node
I20260806 06:41:26.379557 15344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:26.379808 15344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:26.380973 15344 hybrid_clock.cc:648] HybridClock initialized: now 1785998486380968 us; error 37 us; skew 500 ppm
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:26.383384 15344 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:26.383814 15344 webserver.cc:460] Webserver started at http://127.14.51.66:43595/ using document root <none> and password file <none>
I20260806 06:41:26.384032 15344 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:26.384123 15344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:26.384276 15344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:26.385280 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "46ef6fed24224610ae86afc2a2b7e1a5"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "1877f4a994b7bc704ecf139b0f79e718"
server_key_iv: "871fd8be30778f3918ab83cc5382c0af"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.385635 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "46ef6fed24224610ae86afc2a2b7e1a5"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "1877f4a994b7bc704ecf139b0f79e718"
server_key_iv: "871fd8be30778f3918ab83cc5382c0af"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.386945 15344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:26.387715 15359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.387889 15344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.387985 15344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "46ef6fed24224610ae86afc2a2b7e1a5"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "1877f4a994b7bc704ecf139b0f79e718"
server_key_iv: "871fd8be30778f3918ab83cc5382c0af"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.388075 15344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:26.397667 15344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:26.399725 15344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:26.399878 15344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:26.400096 15344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:26.400382 15344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:26.400439 15344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.400477 15344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:26.400504 15344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.406569 15344 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:38959
I20260806 06:41:26.406639 15479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:38959 every 8 connection(s)
I20260806 06:41:26.406935 15344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:26.414888 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15344
I20260806 06:41:26.414985 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:26.415201 14541 external_mini_cluster.cc:1428] Setting key 325dde83be9d965a64e539b12553cd32
I20260806 06:41:26.416157 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.407694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44267 (local address 127.14.51.126:40277)
0806 06:41:26.407754 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.407756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.407951 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:41:26.408864 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.408866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.408877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.408942 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.409496 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.409983 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.410917 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.411098 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.411236 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.413171 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.413172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.413173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.413179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.414646 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.415169 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.415170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.415171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.415230 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.415502 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.415503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.415503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.415620 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.415693 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.415964 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.416067 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:41:26.416293 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.407676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40277 (local address 127.14.51.66:44267)
0806 06:41:26.407835 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.407872 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.408746 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.408991 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.408994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.409230 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:26.409401 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.409405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.410073 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.410074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.410841 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.410842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.411241 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.411242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.411334 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.411773 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:26.411780 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:26.413086 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:26.414730 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.414737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.414739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.415060 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.415302 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.415303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.415303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.415430 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.415699 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:26.415705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:26.415833 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.415931 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.416080 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260806 06:41:26.417609 15480 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40277
I20260806 06:41:26.417698 15480 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:26.417886 15480 heartbeater.cc:511] Master 127.14.51.126:40277 requested a full tablet report, sending...
I20260806 06:41:26.418267 15139 ts_manager.cc:194] Registered new tserver with Master: 46ef6fed24224610ae86afc2a2b7e1a5 (127.14.51.66:38959)
I20260806 06:41:26.418644 15139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:44267
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:26.441499 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:26.520916 15485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:26.521132 15485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:26.521170 15485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:26.522343 15485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:26.522415 15485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:26.522468 15485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:26.523988 15485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40277
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:26.524228 15485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:26.524442 15485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:26.529290 15492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:26.529317 15485 server_base.cc:1034] running on GCE node
W20260806 06:41:26.529261 15491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:26.529271 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:26.529844 15485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:26.530122 15485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:26.531284 15485 hybrid_clock.cc:648] HybridClock initialized: now 1785998486531277 us; error 40 us; skew 500 ppm
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:26.533641 15485 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:26.534119 15485 webserver.cc:460] Webserver started at http://127.14.51.67:42511/ using document root <none> and password file <none>
I20260806 06:41:26.534339 15485 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:26.534406 15485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:26.534540 15485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:26.535578 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "bec15328e00f4d8191121481da78af12"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "ce5c1ec3f7afffc849616cd272b60077"
server_key_iv: "8255677b51dc00ed45985aeec3f7d1ce"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.535928 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "bec15328e00f4d8191121481da78af12"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "ce5c1ec3f7afffc849616cd272b60077"
server_key_iv: "8255677b51dc00ed45985aeec3f7d1ce"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.537259 15485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:26.537961 15503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.538132 15485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:26.538226 15485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "bec15328e00f4d8191121481da78af12"
format_stamp: "Formatted at 2026-08-06 06:41:26 on dist-test-slave-1xrh"
server_key: "ce5c1ec3f7afffc849616cd272b60077"
server_key_iv: "8255677b51dc00ed45985aeec3f7d1ce"
server_key_version: "encryptionkey@0"
I20260806 06:41:26.538304 15485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:26.553817 15485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:26.555576 15485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:26.555730 15485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:26.555943 15485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:26.556239 15485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:26.556293 15485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.556334 15485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:26.556362 15485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:26.562440 15485 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:44137
I20260806 06:41:26.562494 15622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:44137 every 8 connection(s)
I20260806 06:41:26.562808 15485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:26.565936 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15485
I20260806 06:41:26.566007 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:26.566167 14541 external_mini_cluster.cc:1428] Setting key e47634e9dd85d5e2634b46f8589c2a5d
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:26.572086 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.563482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:41531 (local address 127.14.51.126:40277)
0806 06:41:26.563540 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.563542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.563731 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:41:26.564685 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.564686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.564700 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.564764 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.565350 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.565866 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.567299 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.567496 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.567573 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.569544 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.569545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.569545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.569553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.570703 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.571106 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.571107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.571107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.571193 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.571443 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.571444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.571445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.571567 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.571645 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.571906 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.571986 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:41:26.572253 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.563469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40277 (local address 127.14.51.67:41531)
0806 06:41:26.563620 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.563653 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.564573 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.564836 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.564839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.565081 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:26.565251 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.565255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.565966 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.565967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.567215 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.567218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.567581 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.567583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.567714 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.568139 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:26.568147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:26.569443 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:26.570788 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.570795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.570797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.571016 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.571259 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:26.571260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:26.571261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:26.571372 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:26.571635 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:26.571641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:26.571774 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.571881 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.572029 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260806 06:41:26.572700 15624 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40277
I20260806 06:41:26.572793 15624 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:26.572963 15624 heartbeater.cc:511] Master 127.14.51.126:40277 requested a full tablet report, sending...
I20260806 06:41:26.573303 15139 ts_manager.cc:194] Registered new tserver with Master: bec15328e00f4d8191121481da78af12 (127.14.51.67:44137)
I20260806 06:41:26.573673 15139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:41531
I20260806 06:41:26.577948 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:41:26.585016 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.579792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.14.51.126:40277)
0806 06:41:26.579866 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.579868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.579901 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:41:26.579953 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.579954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.579961 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.580004 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.580482 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.580911 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.581561 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.581728 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.581774 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.582855 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.582856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.582856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.582860 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.584097 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.584398 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.584399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.584399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.584446 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.584603 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.584603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.584604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.584724 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.584796 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.584879 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.584934 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:41:26.587389 15139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:26.589236 15139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:26.596541 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:44137 (local address 127.0.0.1:40348)
0806 06:41:26.592456 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.592492 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.592560 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.592922 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.592924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.592926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:26.592978 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.592979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.594472 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.594473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.595563 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.595565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.596211 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.596212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.596313 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.596330 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.596397 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.596457 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":60,"threads_started":1}
I20260806 06:41:26.596544 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:46533 (local address 127.0.0.1:58650)
0806 06:41:26.592662 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.592673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.592713 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.593061 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.593062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.593064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:26.593115 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.593117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.594184 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.594185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.595474 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.595477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.596345 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.596346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.596410 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.596413 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.596458 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.596485 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":26,"thread_start_us":57,"threads_started":1}
I20260806 06:41:26.596608 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.14.51.65:46533)
0806 06:41:26.592882 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.592886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.592897 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:26.592937 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.592940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.592955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.593217 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:26.593309 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.594077 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.595565 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.596272 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.596347 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.596405 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.596484 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.596529 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":168,"threads_started":1}
I20260806 06:41:26.596911 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.14.51.67:44137)
0806 06:41:26.592677 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.592682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.592692 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:26.592729 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.592731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.592750 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.592834 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:26.593075 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.594355 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.595628 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.596107 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.596209 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.596269 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.596763 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.596808 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
I20260806 06:41:26.597357 15269 tablet_service.cc:1467] Processing CreateTablet for tablet f08b8a80e5a64ce1988bce48c030f6b6 (DEFAULT_TABLE table=test-table [id=6bc18e033d0f4864a6d304a2c5f3effd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:26.597718 15269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08b8a80e5a64ce1988bce48c030f6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:26.597774 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:38959 (local address 127.0.0.1:50508)
0806 06:41:26.592947 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.592957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.593019 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.593195 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.593196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.593198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:26.593238 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.593240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.595219 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.595220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.596337 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.596340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.597524 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.597525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.597586 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.597590 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.597654 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.597697 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":63,"threads_started":1}
I20260806 06:41:26.597932 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.592526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50508 (local address 127.14.51.66:38959)
0806 06:41:26.592791 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.592795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.593040 (+ 245us) server_negotiation.cc:407] Connection header received
0806 06:41:26.593087 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.593089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.593106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.593186 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:26.593298 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.594377 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.596424 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.597410 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.597504 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.597560 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.597719 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.597759 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":150,"threads_started":1}
I20260806 06:41:26.598484 15408 tablet_service.cc:1467] Processing CreateTablet for tablet f08b8a80e5a64ce1988bce48c030f6b6 (DEFAULT_TABLE table=test-table [id=6bc18e033d0f4864a6d304a2c5f3effd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:26.598805 15408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08b8a80e5a64ce1988bce48c030f6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:26.600692 15638 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Bootstrap starting.
I20260806 06:41:26.601146 15555 tablet_service.cc:1467] Processing CreateTablet for tablet f08b8a80e5a64ce1988bce48c030f6b6 (DEFAULT_TABLE table=test-table [id=6bc18e033d0f4864a6d304a2c5f3effd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:26.601436 15555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08b8a80e5a64ce1988bce48c030f6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:26.601876 15638 tablet_bootstrap.cc:654] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:26.601995 15639 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Bootstrap starting.
I20260806 06:41:26.602841 15638 log.cc:824] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:26.602962 15639 tablet_bootstrap.cc:654] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:26.603621 15638 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: No bootstrap required, opened a new log
I20260806 06:41:26.603703 15638 ts_tablet_manager.cc:1397] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:26.603899 15639 log.cc:824] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:26.604936 15639 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: No bootstrap required, opened a new log
I20260806 06:41:26.605055 15639 ts_tablet_manager.cc:1397] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:41:26.605528 15642 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Bootstrap starting.
I20260806 06:41:26.606357 15638 raft_consensus.cc:348] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.606647 15638 raft_consensus.cc:374] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:26.606705 15638 raft_consensus.cc:729] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14e184b5f4c4ee0ba7b2a5a9132947c, State: Initialized, Role: FOLLOWER
I20260806 06:41:26.606832 15638 consensus_queue.cc:260] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.607007 15642 tablet_bootstrap.cc:654] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:26.607331 15338 heartbeater.cc:503] Master 127.14.51.126:40277 was elected leader, sending a full tablet report...
I20260806 06:41:26.607285 15638 ts_tablet_manager.cc:1428] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:41:26.607620 15639 raft_consensus.cc:348] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.607777 15639 raft_consensus.cc:374] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:26.607833 15639 raft_consensus.cc:729] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ef6fed24224610ae86afc2a2b7e1a5, State: Initialized, Role: FOLLOWER
I20260806 06:41:26.607959 15639 consensus_queue.cc:260] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.608325 15642 log.cc:824] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:26.608345 15480 heartbeater.cc:503] Master 127.14.51.126:40277 was elected leader, sending a full tablet report...
I20260806 06:41:26.608368 15639 ts_tablet_manager.cc:1428] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:41:26.609066 15642 tablet_bootstrap.cc:492] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: No bootstrap required, opened a new log
I20260806 06:41:26.609155 15642 ts_tablet_manager.cc:1397] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 06:41:26.610961 15642 raft_consensus.cc:348] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.611085 15642 raft_consensus.cc:374] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:26.611106 15642 raft_consensus.cc:729] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec15328e00f4d8191121481da78af12, State: Initialized, Role: FOLLOWER
I20260806 06:41:26.611253 15642 consensus_queue.cc:260] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.611522 15642 ts_tablet_manager.cc:1428] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:41:26.611572 15624 heartbeater.cc:503] Master 127.14.51.126:40277 was elected leader, sending a full tablet report...
W20260806 06:41:26.657632 15481 tablet.cc:2369] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:26.759931 15339 tablet.cc:2369] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:26.791638 15643 raft_consensus.cc:482] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:26.791760 15643 raft_consensus.cc:504] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.792248 15643 leader_election.cc:288] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bec15328e00f4d8191121481da78af12 (127.14.51.67:44137), 46ef6fed24224610ae86afc2a2b7e1a5 (127.14.51.66:38959)
I20260806 06:41:26.795477 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.792527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:56539 (local address 127.14.51.67:44137)
0806 06:41:26.792595 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.792599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.792698 (+ 99us) server_negotiation.cc:407] Connection header received
0806 06:41:26.792763 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.792765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.792777 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.792850 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:26.793031 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.793524 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.794709 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.795087 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.795194 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.795207 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.795357 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.795396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:41:26.795463 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.792423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:44137 (local address 127.14.51.65:56539)
0806 06:41:26.792600 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.792613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.792681 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.792868 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.792869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.792871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:26.792953 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.792955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.793612 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.793613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.794589 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.794591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.795200 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.795201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.795276 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.795280 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.795334 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.795363 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":51,"thread_start_us":110,"threads_started":1}
I20260806 06:41:26.795588 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.792560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:35841 (local address 127.14.51.66:38959)
0806 06:41:26.792604 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.792606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.792720 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:41:26.792780 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.792781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.792791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.792859 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:26.793094 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.793529 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.794646 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.795186 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.795255 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.795266 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.795474 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.795515 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:26.795619 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.792425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:38959 (local address 127.14.51.65:35841)
0806 06:41:26.792617 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:26.792629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:26.792717 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:26.792865 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:26.792866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:26.792867 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:26.793020 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.793022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.793614 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.793615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.794559 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:26.794561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.795263 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.795264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:26.795380 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.795384 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:26.795444 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:41:26.795480 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":52,"thread_start_us":104,"threads_started":1}
I20260806 06:41:26.796020 15430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08b8a80e5a64ce1988bce48c030f6b6" candidate_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" is_pre_election: true
I20260806 06:41:26.796030 15575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08b8a80e5a64ce1988bce48c030f6b6" candidate_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec15328e00f4d8191121481da78af12" is_pre_election: true
I20260806 06:41:26.796166 15430 raft_consensus.cc:2454] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14e184b5f4c4ee0ba7b2a5a9132947c in term 0.
I20260806 06:41:26.796201 15575 raft_consensus.cc:2454] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14e184b5f4c4ee0ba7b2a5a9132947c in term 0.
I20260806 06:41:26.796600 15221 leader_election.cc:302] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46ef6fed24224610ae86afc2a2b7e1a5, d14e184b5f4c4ee0ba7b2a5a9132947c; no voters:
I20260806 06:41:26.796736 15643 raft_consensus.cc:2781] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:26.796789 15643 raft_consensus.cc:482] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:26.796806 15643 raft_consensus.cc:3037] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:26.797572 15643 raft_consensus.cc:504] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.797755 15643 leader_election.cc:288] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [CANDIDATE]: Term 1 election: Requested vote from peers bec15328e00f4d8191121481da78af12 (127.14.51.67:44137), 46ef6fed24224610ae86afc2a2b7e1a5 (127.14.51.66:38959)
I20260806 06:41:26.797902 15575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08b8a80e5a64ce1988bce48c030f6b6" candidate_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec15328e00f4d8191121481da78af12"
I20260806 06:41:26.797943 15430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08b8a80e5a64ce1988bce48c030f6b6" candidate_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46ef6fed24224610ae86afc2a2b7e1a5"
I20260806 06:41:26.798000 15575 raft_consensus.cc:3037] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:26.798004 15430 raft_consensus.cc:3037] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:26.798815 15575 raft_consensus.cc:2454] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14e184b5f4c4ee0ba7b2a5a9132947c in term 1.
I20260806 06:41:26.798949 15430 raft_consensus.cc:2454] T f08b8a80e5a64ce1988bce48c030f6b6 P 46ef6fed24224610ae86afc2a2b7e1a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14e184b5f4c4ee0ba7b2a5a9132947c in term 1.
I20260806 06:41:26.799063 15222 leader_election.cc:302] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bec15328e00f4d8191121481da78af12, d14e184b5f4c4ee0ba7b2a5a9132947c; no voters:
I20260806 06:41:26.799176 15643 raft_consensus.cc:2781] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:26.799343 15643 raft_consensus.cc:686] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [term 1 LEADER]: Becoming Leader. State: Replica: d14e184b5f4c4ee0ba7b2a5a9132947c, State: Running, Role: LEADER
I20260806 06:41:26.799451 15643 consensus_queue.cc:237] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } }
I20260806 06:41:26.800397 15138 catalog_manager.cc:5495] T f08b8a80e5a64ce1988bce48c030f6b6 P d14e184b5f4c4ee0ba7b2a5a9132947c reported cstate change: term changed from 0 to 1, leader changed from <none> to d14e184b5f4c4ee0ba7b2a5a9132947c (127.14.51.65). New cstate: current_term: 1 leader_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d14e184b5f4c4ee0ba7b2a5a9132947c" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bec15328e00f4d8191121481da78af12" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46ef6fed24224610ae86afc2a2b7e1a5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38959 } health_report { overall_health: UNKNOWN } } }
W20260806 06:41:26.813320 15625 tablet.cc:2369] T f08b8a80e5a64ce1988bce48c030f6b6 P bec15328e00f4d8191121481da78af12: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:26.826368 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.819091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.14.51.65:46533)
0806 06:41:26.819185 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.819187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.819365 (+ 178us) server_negotiation.cc:407] Connection header received
0806 06:41:26.819408 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.819409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.819420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.819470 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.820111 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.820566 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.821254 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.821424 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.821471 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.824003 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.824012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.824012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.824019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.825407 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.825724 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.825727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.825727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.825779 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.825946 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.825947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.825947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.826087 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.826188 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.826249 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.826307 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:26.851284 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.844262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.14.51.65:46533)
0806 06:41:26.844354 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.844356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.844440 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:41:26.844486 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.844487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.844496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.844548 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.845262 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.845728 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.846520 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.846699 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.846751 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.848890 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.848892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.848892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.848898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.850299 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.850585 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.850585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.850585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.850635 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.850777 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.850777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.850777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.850854 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.850937 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.851053 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.851115 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:26.873512 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:26.867203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.14.51.65:46533)
0806 06:41:26.867274 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:26.867276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:26.867285 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:26.867345 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:26.867346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:26.867356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:26.867405 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:26.867950 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.868417 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.869138 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:26.869340 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:26.869425 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:26.871514 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:26.871515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:26.871516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:26.871521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:26.872647 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.872944 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.872945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.872945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.872992 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:26.873137 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:26.873137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:26.873137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:26.873208 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:26.873307 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:26.873375 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:41:26.873444 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 06:41:26.874188 15248 tablet_service.cc:2908] Rejecting scan request for tablet f08b8a80e5a64ce1988bce48c030f6b6: Uninitialized: clean time has not yet been initialized
I20260806 06:41:26.875243 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15204
I20260806 06:41:26.880779 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15344
I20260806 06:41:26.886410 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15485
I20260806 06:41:26.893045 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15105
2026-08-06T06:41:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2162 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15663](info): set up 2 sockets
Aug 06 06:41:26 dist-test-slave-1xrh krb5kdc[15663](info): commencing operation
krb5kdc: starting...
W20260806 06:41:28.354249 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.440s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:28 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:28.535322 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37539
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:28.681365 15676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:28.681829 15676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:28.681885 15676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:28.684237 15676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:28.684346 15676 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:41:28.684401 15676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:28.684496 15676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:28.684554 15676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:28.684633 15676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:28.687304 15676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:37539
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:28.687813 15676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:28.688248 15676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:28.696524 15682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:28.696522 15684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:28.696524 15681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:28.697588 15676 server_base.cc:1034] running on GCE node
I20260806 06:41:28.698091 15676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:28.698688 15676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:28.699894 15676 hybrid_clock.cc:648] HybridClock initialized: now 1785998488699857 us; error 50 us; skew 500 ppm
Aug 06 06:41:28 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998488, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:28.709717 15676 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:28.710626 15676 webserver.cc:460] Webserver started at http://127.14.51.126:42361/ using document root <none> and password file <none>
I20260806 06:41:28.711225 15676 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:28.711333 15676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:28.711673 15676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:28.713644 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42"
format_stamp: "Formatted at 2026-08-06 06:41:28 on dist-test-slave-1xrh"
server_key: "2bad0a7b53ce8c87709544837f5e7909"
server_key_iv: "8e9f6b5e043a57ae4afef43d5cce684f"
server_key_version: "encryptionkey@0"
I20260806 06:41:28.714357 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42"
format_stamp: "Formatted at 2026-08-06 06:41:28 on dist-test-slave-1xrh"
server_key: "2bad0a7b53ce8c87709544837f5e7909"
server_key_iv: "8e9f6b5e043a57ae4afef43d5cce684f"
server_key_version: "encryptionkey@0"
I20260806 06:41:28.716917 15676 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260806 06:41:28.718159 15692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:28.718427 15676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:28.718590 15676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42"
format_stamp: "Formatted at 2026-08-06 06:41:28 on dist-test-slave-1xrh"
server_key: "2bad0a7b53ce8c87709544837f5e7909"
server_key_iv: "8e9f6b5e043a57ae4afef43d5cce684f"
server_key_version: "encryptionkey@0"
I20260806 06:41:28.718703 15676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:28.742077 15676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:28.747416 15676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:28.747699 15676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:28.760167 15676 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:37539
I20260806 06:41:28.760305 15755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:37539 every 8 connection(s)
I20260806 06:41:28.761173 15676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:28.762804 15756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:28.763741 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15676
I20260806 06:41:28.763931 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:28.764292 14541 external_mini_cluster.cc:1428] Setting key 0187205179e4a6ad5abf6ea955745323
I20260806 06:41:28.770300 15756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Bootstrap starting.
I20260806 06:41:28.772282 15756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:28.773003 15756 log.cc:824] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:28.774566 15756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: No bootstrap required, opened a new log
Aug 06 06:41:28 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:28.781406 15756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } }
I20260806 06:41:28.781862 15756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:28.782017 15756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb0d8d0f9724b23a7aa8369b0b3be42, State: Initialized, Role: FOLLOWER
I20260806 06:41:28.782285 15756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } }
I20260806 06:41:28.782635 15756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:28.782734 15756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:28.782831 15756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:28.784534 15756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } }
I20260806 06:41:28.784782 15756 leader_election.cc:302] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bb0d8d0f9724b23a7aa8369b0b3be42; no voters:
I20260806 06:41:28.785238 15756 leader_election.cc:288] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:28.785355 15761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:28.785760 15761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [term 1 LEADER]: Becoming Leader. State: Replica: 5bb0d8d0f9724b23a7aa8369b0b3be42, State: Running, Role: LEADER
I20260806 06:41:28.785835 15756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:28.786214 15761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } }
I20260806 06:41:28.787225 15761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb0d8d0f9724b23a7aa8369b0b3be42. Latest consensus state: current_term: 1 leader_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } } }
I20260806 06:41:28.787467 15761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:28.787806 15762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb0d8d0f9724b23a7aa8369b0b3be42" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37539 } } }
I20260806 06:41:28.787971 15762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:28.788282 15771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:28.789616 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:28.765805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.14.51.126:37539)
0806 06:41:28.766107 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:28.766125 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:28.766202 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:41:28.767448 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:28.767469 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:28.767550 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:28.767911 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:28.768873 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:28.770350 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:28.771822 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:28.773107 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:28.773296 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:28.779800 (+ 6504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:28.779832 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:28.779836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:28.779898 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:28.784563 (+ 4665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:28.785702 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:28.785707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:28.785707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:28.785924 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:28.786621 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:28.786622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:28.786623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:28.787290 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:28.787648 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:28.788829 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0806 06:41:28.789220 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":141,"threads_started":1}
I20260806 06:41:28.790337 15771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:28.793421 15771 catalog_manager.cc:1347] Generated new cluster ID: 334b4c76726b4e8592691dc4fb3b1aac
I20260806 06:41:28.793524 15771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:28.822190 15771 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:28.823519 15771 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:28.841702 15771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Generated new TSK 0
I20260806 06:41:28.841985 15771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:28.891404 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37539
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:29.040503 15783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:29.040926 15783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:29.041025 15783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:29.042933 15783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:29.043025 15783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:29.043087 15783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:29.043183 15783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:29.045513 15783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:29.045912 15783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:29.046471 15783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:29.055047 15788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:29.055048 15791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:29.055272 15789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:29.055754 15783 server_base.cc:1034] running on GCE node
I20260806 06:41:29.056100 15783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:29.056394 15783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:29.057607 15783 hybrid_clock.cc:648] HybridClock initialized: now 1785998489057566 us; error 58 us; skew 500 ppm
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:29.062191 15783 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:29.062975 15783 webserver.cc:460] Webserver started at http://127.14.51.65:40679/ using document root <none> and password file <none>
I20260806 06:41:29.063417 15783 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:29.063555 15783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:29.063895 15783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:29.065809 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "aedac398ae97e848f58654b9fbcb8544"
server_key_iv: "eb97e41ee00bf61181541ffd825c898f"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.066576 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "aedac398ae97e848f58654b9fbcb8544"
server_key_iv: "eb97e41ee00bf61181541ffd825c898f"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.069798 15783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 06:41:29.071589 15798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.071868 15783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:29.072029 15783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "aedac398ae97e848f58654b9fbcb8544"
server_key_iv: "eb97e41ee00bf61181541ffd825c898f"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.072146 15783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:29.105865 15783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:29.109702 15783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:29.110028 15783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:29.111361 15783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:29.111464 15783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.111593 15783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:29.111702 15783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:29.128875 15783 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44081
I20260806 06:41:29.129171 15915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44081 every 8 connection(s)
I20260806 06:41:29.129873 15783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:29.131332 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15783
I20260806 06:41:29.131580 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:29.131984 14541 external_mini_cluster.cc:1428] Setting key 84f0e9b284bdc262dfac7e93d1e1af6e
I20260806 06:41:29.135397 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.112349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:49475 (local address 127.14.51.126:37539)
0806 06:41:29.112557 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.112571 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.113140 (+ 569us) server_negotiation.cc:407] Connection header received
0806 06:41:29.114185 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.114187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.114226 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.114355 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.116058 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.117610 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.119531 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.120304 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.120443 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.128439 (+ 7996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.128441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.128449 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.128459 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.132147 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.133162 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.133163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.133163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.133259 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.133925 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.133926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.133926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.134154 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.134332 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.135017 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.135243 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 06:41:29.136052 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.112448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.65:49475)
0806 06:41:29.112768 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.112883 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.113907 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.114555 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.114566 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.115408 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.115881 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.115888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.117947 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.117948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.119433 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.119436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.120474 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.120476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.120702 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.122255 (+ 1553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.122288 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.128184 (+ 5896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.132292 (+ 4108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.132299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.132301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.133008 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.133417 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.133418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.133418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.133654 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.134320 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.134327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.134619 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.134939 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.135349 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":243,"threads_started":1}
I20260806 06:41:29.136862 15916 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37539
I20260806 06:41:29.137038 15916 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:29.137421 15916 heartbeater.cc:511] Master 127.14.51.126:37539 requested a full tablet report, sending...
I20260806 06:41:29.138374 15708 ts_manager.cc:194] Registered new tserver with Master: 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65:44081)
I20260806 06:41:29.139817 15708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:49475
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
I20260806 06:41:29.150695 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.139224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:36609 (local address 127.14.51.126:37539)
0806 06:41:29.139292 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.139294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.139598 (+ 304us) server_negotiation.cc:407] Connection header received
0806 06:41:29.139768 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.139779 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.139831 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.139976 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.140518 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.141743 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.143173 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.143459 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.143645 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.145675 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.145686 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.145686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.145710 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.148470 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.149254 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.149255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.149256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.149376 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.149714 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.149715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.149715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.149897 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.150085 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.150338 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.150468 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:41:29.150754 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.139168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.65:36609)
0806 06:41:29.139405 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.139423 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.139654 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.139980 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.139980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.140268 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.140353 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.140356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.141907 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.141907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.142856 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.142861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.143626 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.143637 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.143870 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.144640 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.144649 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.145446 (+ 797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.148631 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.148632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.148632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.149037 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.149487 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.149488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.149489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.149573 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.150018 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.150019 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.150084 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.150303 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.150478 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:29.175642 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37539
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:29.342834 15926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:29.343021 15926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:29.343045 15926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:29.344264 15926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:29.344312 15926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:29.344333 15926 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:29.344350 15926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:29.345685 15926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:29.345865 15926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:29.346030 15926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:29.350746 15932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:29.350813 15926 server_base.cc:1034] running on GCE node
W20260806 06:41:29.350757 15931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:29.350831 15934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:29.351213 15926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:29.351465 15926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:29.352684 15926 hybrid_clock.cc:648] HybridClock initialized: now 1785998489352681 us; error 39 us; skew 500 ppm
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:29.355370 15926 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:29.355773 15926 webserver.cc:460] Webserver started at http://127.14.51.66:45029/ using document root <none> and password file <none>
I20260806 06:41:29.356019 15926 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:29.356125 15926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:29.356279 15926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:29.357285 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "640ec919a3b84cb5be22db15f108f0f5"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "06b8decbf99cfc066961d1daf14bb4db"
server_key_iv: "a45071a2d3982a63ea03b2080275ba07"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.357642 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "640ec919a3b84cb5be22db15f108f0f5"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "06b8decbf99cfc066961d1daf14bb4db"
server_key_iv: "a45071a2d3982a63ea03b2080275ba07"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.358932 15926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:29.359777 15941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.359961 15926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:29.360059 15926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "640ec919a3b84cb5be22db15f108f0f5"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "06b8decbf99cfc066961d1daf14bb4db"
server_key_iv: "a45071a2d3982a63ea03b2080275ba07"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.360146 15926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:29.375293 15926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:29.376963 15926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:29.377116 15926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:29.377562 15926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:29.377624 15926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.377665 15926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:29.377712 15926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:29.385074 15926 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:45961
I20260806 06:41:29.385159 16057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:45961 every 8 connection(s)
I20260806 06:41:29.385433 15926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:29.388502 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.378011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:55381 (local address 127.14.51.126:37539)
0806 06:41:29.378083 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.378086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.378263 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:41:29.378691 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.378692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.378707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.378790 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.379485 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.380321 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.381991 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.382183 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.382273 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.384954 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.384955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.384956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.384970 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.387040 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.387545 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.387546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.387547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.387604 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.387840 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.387840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.387841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.387988 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.388060 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.388345 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.388413 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:41:29.388645 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.377986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.66:55381)
0806 06:41:29.378137 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.378173 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.378601 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.378835 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.378837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.379178 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.379357 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.379361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.380493 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.380513 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.381901 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.381905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.382291 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.382292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.382428 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.382975 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.382984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.384850 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.387186 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.387191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.387193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.387469 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.387681 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.387681 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.387682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.387773 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.388055 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.388059 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.388160 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.388290 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.388417 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260806 06:41:29.389194 16058 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37539
I20260806 06:41:29.389269 16058 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:29.389438 16058 heartbeater.cc:511] Master 127.14.51.126:37539 requested a full tablet report, sending...
I20260806 06:41:29.389823 15708 ts_manager.cc:194] Registered new tserver with Master: 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961)
I20260806 06:41:29.390192 15708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:55381
I20260806 06:41:29.394471 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 15926
I20260806 06:41:29.394565 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:29.394739 14541 external_mini_cluster.cc:1428] Setting key 2c92f4e1d3b6d62c434bfbf0db619ef1
I20260806 06:41:29.396026 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.390498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:51977 (local address 127.14.51.126:37539)
0806 06:41:29.390531 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.390533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.390542 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:29.390562 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.390563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.390574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.390628 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.391087 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.391605 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.392214 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.392379 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.392427 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.393170 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.393171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.393171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.393176 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.394740 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.395219 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.395220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.395220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.395283 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.395564 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.395565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.395565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.395698 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.395753 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.395916 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.395971 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:29.396168 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.390328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.66:51977)
0806 06:41:29.390453 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.390462 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.390520 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.390634 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.390635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.390832 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.390902 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.390904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.391690 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.391690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.392144 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.392146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.392439 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.392440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.392492 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.392827 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.392830 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.393099 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.394837 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.394838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.394839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.395091 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.395354 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.395355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.395355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.395489 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.395771 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.395772 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.395820 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.395901 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.396015 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:29.420145 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37539
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:29.500308 16068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:29.500491 16068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:29.500514 16068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:29.501551 16068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:29.501585 16068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:29.501603 16068 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:29.501621 16068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:29.502945 16068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:29.503201 16068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:29.503381 16068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:29.508134 16077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:29.508142 16075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:29.508141 16074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:29.508183 16068 server_base.cc:1034] running on GCE node
I20260806 06:41:29.508533 16068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:29.508783 16068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:29.509986 16068 hybrid_clock.cc:648] HybridClock initialized: now 1785998489509973 us; error 33 us; skew 500 ppm
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:29.512496 16068 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:29.512979 16068 webserver.cc:460] Webserver started at http://127.14.51.67:37435/ using document root <none> and password file <none>
I20260806 06:41:29.513203 16068 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:29.513271 16068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:29.513418 16068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:29.514452 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "6f8f52d781c0479281f820f6e0bb3c60"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "084fe2034870c1d71e7b8942a673a21d"
server_key_iv: "aaf3442185437efc54aba4b4205cfb77"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.514813 16068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "6f8f52d781c0479281f820f6e0bb3c60"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "084fe2034870c1d71e7b8942a673a21d"
server_key_iv: "aaf3442185437efc54aba4b4205cfb77"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.516168 16068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:29.516870 16084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.517045 16068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.517150 16068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "6f8f52d781c0479281f820f6e0bb3c60"
format_stamp: "Formatted at 2026-08-06 06:41:29 on dist-test-slave-1xrh"
server_key: "084fe2034870c1d71e7b8942a673a21d"
server_key_iv: "aaf3442185437efc54aba4b4205cfb77"
server_key_version: "encryptionkey@0"
I20260806 06:41:29.517220 16068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:29.533015 16068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:29.535382 16068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:29.535539 16068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:29.536010 16068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:29.536074 16068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.536120 16068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:29.536150 16068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:29.543411 16068 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:38765
I20260806 06:41:29.543509 16204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:38765 every 8 connection(s)
I20260806 06:41:29.543812 16068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:29.544404 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 16068
I20260806 06:41:29.544509 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:29.544682 14541 external_mini_cluster.cc:1428] Setting key 2265c829625aebfd3451a3688c598837
I20260806 06:41:29.546242 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.536545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:53161 (local address 127.14.51.126:37539)
0806 06:41:29.536630 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.536632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.536900 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:41:29.537526 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.537528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.537543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.537621 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.538423 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.539293 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.540292 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.540528 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.540607 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.543423 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.543424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.543425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.543433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.545131 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.545520 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.545521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.545522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.545605 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.545806 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.545806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.545806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.545904 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.545954 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.546125 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.546181 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:41:29.546475 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.536537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.67:53161)
0806 06:41:29.536771 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.536803 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.537383 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.537701 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.537705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.538037 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.538301 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.538307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.539389 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.539391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.540174 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.540176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.540623 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.540624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.540759 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.541369 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.541379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.543330 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.545189 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.545196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.545198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.545435 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.545659 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.545660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.545660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.545752 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.545945 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.545947 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.546040 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.546114 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.546223 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260806 06:41:29.546914 16205 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37539
I20260806 06:41:29.546990 16205 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:29.547192 16205 heartbeater.cc:511] Master 127.14.51.126:37539 requested a full tablet report, sending...
I20260806 06:41:29.547848 15708 ts_manager.cc:194] Registered new tserver with Master: 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765)
I20260806 06:41:29.548331 15708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:53161
I20260806 06:41:29.552596 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.548044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:48117 (local address 127.14.51.126:37539)
0806 06:41:29.548085 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.548087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.548185 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:41:29.548242 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.548242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.548249 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.548294 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.548694 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.549128 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.549753 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.549942 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.549993 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.550700 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.550701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.550701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.550706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.551869 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.552144 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.552145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.552145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.552183 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.552293 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.552293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.552294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.552366 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.552412 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.552499 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.552545 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:41:29.552621 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.547997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.14.51.67:48117)
0806 06:41:29.548123 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.548134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.548191 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.548307 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.548308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.548514 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:29.548582 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.548584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.549214 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.549215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.549681 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.549682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.550006 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.550007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.550062 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.550376 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:29.550380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:29.550628 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:29.551933 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.551934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.551934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.552089 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.552218 (+ 129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:29.552219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:29.552219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:29.552258 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:29.552409 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:29.552409 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:29.552441 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.552484 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.552558 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260806 06:41:29.556318 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:41:29 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:29.578930 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.572773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.14.51.126:37539)
0806 06:41:29.572827 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.572829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.572835 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:41:29.572853 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.572853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.572862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.572902 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:29.573458 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.573912 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.574581 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.574751 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.574798 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.576890 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:29.576891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:29.576891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:29.576897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:29.578042 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.578342 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.578343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.578343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.578389 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:29.578548 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:29.578549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:29.578549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:29.578640 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:29.578709 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.578817 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.578870 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:41:29.580720 15708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:29.582356 15708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:29.589326 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.585092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.14.51.66:45961)
0806 06:41:29.585261 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.585266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.585278 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:41:29.585320 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.585322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.585341 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.585608 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:29.585829 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586513 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588208 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.588876 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588973 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.589038 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.589172 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.589212 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260806 06:41:29.589370 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.584617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44081 (local address 127.0.0.1:43172)
0806 06:41:29.584724 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.584770 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.584853 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.585338 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.585341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.585344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:29.585409 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.585411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586833 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.586834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.588098 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.588102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.588997 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.589075 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.589079 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.589126 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.589274 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":129,"thread_start_us":38,"threads_started":1}
I20260806 06:41:29.589378 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.584989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45961 (local address 127.0.0.1:60400)
0806 06:41:29.585097 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.585106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.585145 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.585682 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.585682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.585684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:29.585727 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.585729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586658 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.586659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.588099 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.588102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.588977 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.589061 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.589076 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.589152 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.589201 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":34,"thread_start_us":77,"threads_started":1}
I20260806 06:41:29.589485 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.584749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.14.51.65:44081)
0806 06:41:29.585148 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.585153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.585164 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:29.585208 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.585210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.585225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.585526 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:29.585593 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586755 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588174 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.588889 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.589185 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.589292 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.589320 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.589362 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":66,"threads_started":1}
I20260806 06:41:29.589901 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.584807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:38765 (local address 127.0.0.1:42194)
0806 06:41:29.584923 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.584933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.584979 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.585474 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.585475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.585476 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:29.585518 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.585519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586843 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.586844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.587993 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.587996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.589005 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.589006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.589065 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.589085 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.589126 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.589197 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260806 06:41:29.590401 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.584919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.14.51.67:38765)
0806 06:41:29.585316 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.585320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.585328 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:29.585363 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.585367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.585381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.585465 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:29.585604 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.586715 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.588084 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.588889 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.590141 (+ 1252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.590188 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.590227 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.590271 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":89,"threads_started":1}
I20260806 06:41:29.590371 15991 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d (DEFAULT_TABLE table=test-table [id=aff0a47c0b1546dc8eff19e74357bddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.590757 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2e13cfc6924adfac0b72e97bf40b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.590930 16134 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d (DEFAULT_TABLE table=test-table [id=aff0a47c0b1546dc8eff19e74357bddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.591251 16134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2e13cfc6924adfac0b72e97bf40b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.593775 16228 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Bootstrap starting.
I20260806 06:41:29.594218 16229 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Bootstrap starting.
I20260806 06:41:29.594612 15849 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d (DEFAULT_TABLE table=test-table [id=aff0a47c0b1546dc8eff19e74357bddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.594934 16229 tablet_bootstrap.cc:654] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.594964 15849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2e13cfc6924adfac0b72e97bf40b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.595067 16228 tablet_bootstrap.cc:654] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.595345 16229 log.cc:824] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:29.595448 16228 log.cc:824] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:29.596230 16228 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: No bootstrap required, opened a new log
I20260806 06:41:29.596303 16228 ts_tablet_manager.cc:1397] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:41:29.598380 16232 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Bootstrap starting.
I20260806 06:41:29.598446 16228 raft_consensus.cc:348] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.598626 16228 raft_consensus.cc:374] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.598668 16228 raft_consensus.cc:729] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8f52d781c0479281f820f6e0bb3c60, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.598783 16228 consensus_queue.cc:260] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.598945 16229 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: No bootstrap required, opened a new log
I20260806 06:41:29.599018 16229 ts_tablet_manager.cc:1397] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 06:41:29.599062 16205 heartbeater.cc:503] Master 127.14.51.126:37539 was elected leader, sending a full tablet report...
I20260806 06:41:29.599045 16228 ts_tablet_manager.cc:1428] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:29.599246 16232 tablet_bootstrap.cc:654] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.599581 16232 log.cc:824] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:29.600564 16232 tablet_bootstrap.cc:492] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: No bootstrap required, opened a new log
I20260806 06:41:29.600618 16232 ts_tablet_manager.cc:1397] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:41:29.601140 16229 raft_consensus.cc:348] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.601281 16229 raft_consensus.cc:374] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.601315 16229 raft_consensus.cc:729] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640ec919a3b84cb5be22db15f108f0f5, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.601425 16229 consensus_queue.cc:260] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.601693 16229 ts_tablet_manager.cc:1428] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:41:29.601727 16058 heartbeater.cc:503] Master 127.14.51.126:37539 was elected leader, sending a full tablet report...
I20260806 06:41:29.602733 16232 raft_consensus.cc:348] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.602864 16232 raft_consensus.cc:374] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.602906 16232 raft_consensus.cc:729] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068e7ca7b8bc43ee9e4b27bec85b5acd, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.603008 16232 consensus_queue.cc:260] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.603266 16232 ts_tablet_manager.cc:1428] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:41:29.603303 15916 heartbeater.cc:503] Master 127.14.51.126:37539 was elected leader, sending a full tablet report...
W20260806 06:41:29.630990 15917 tablet.cc:2369] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:29.636086 16059 tablet.cc:2369] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:29.639284 16237 raft_consensus.cc:482] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:29.639393 16237 raft_consensus.cc:504] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.639719 16237 leader_election.cc:288] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765), 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961)
I20260806 06:41:29.643601 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.640169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43129 (local address 127.14.51.66:45961)
0806 06:41:29.640209 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.640211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.640445 (+ 234us) server_negotiation.cc:407] Connection header received
0806 06:41:29.640483 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.640484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.640493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.640541 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:29.640675 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.641362 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.642685 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.643253 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.643328 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.643336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.643499 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.643535 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:29.643601 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.639990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:33321 (local address 127.14.51.67:38765)
0806 06:41:29.640053 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.640070 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.640081 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:29.640167 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.640167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.640177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.640229 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:29.640472 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.641253 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.642687 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.643253 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.643328 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.643337 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.643499 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.643535 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:41:29.643659 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.639889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:38765 (local address 127.14.51.65:33321)
0806 06:41:29.640005 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.640017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.640078 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.640239 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.640239 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.640241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:29.640388 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.640391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.641329 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.641330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.642596 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.642599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.643335 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.643337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.643416 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.643422 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.643481 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.643550 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":55,"thread_start_us":76,"threads_started":1}
I20260806 06:41:29.643668 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.640083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45961 (local address 127.14.51.65:43129)
0806 06:41:29.640218 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:29.640227 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:29.640429 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:29.640555 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:29.640555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:29.640557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:29.640610 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.640612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.641432 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.641433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.642596 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:29.642599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.643337 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.643338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:29.643416 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.643421 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:29.643480 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:41:29.643543 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":11,"thread_start_us":93,"threads_started":1}
I20260806 06:41:29.644078 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2e13cfc6924adfac0b72e97bf40b9d" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60" is_pre_election: true
I20260806 06:41:29.644234 16155 raft_consensus.cc:2454] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 0.
I20260806 06:41:29.644518 15801 leader_election.cc:302] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068e7ca7b8bc43ee9e4b27bec85b5acd, 6f8f52d781c0479281f820f6e0bb3c60; no voters:
I20260806 06:41:29.644593 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2e13cfc6924adfac0b72e97bf40b9d" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640ec919a3b84cb5be22db15f108f0f5" is_pre_election: true
I20260806 06:41:29.644661 16237 raft_consensus.cc:2781] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:29.644721 16237 raft_consensus.cc:482] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:29.644753 16237 raft_consensus.cc:3037] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.644744 16011 raft_consensus.cc:2454] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 0.
I20260806 06:41:29.645498 16237 raft_consensus.cc:504] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.645630 16237 leader_election.cc:288] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 election: Requested vote from peers 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765), 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961)
I20260806 06:41:29.645797 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2e13cfc6924adfac0b72e97bf40b9d" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60"
I20260806 06:41:29.645810 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2e13cfc6924adfac0b72e97bf40b9d" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640ec919a3b84cb5be22db15f108f0f5"
I20260806 06:41:29.645881 16155 raft_consensus.cc:3037] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.645881 16011 raft_consensus.cc:3037] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.646801 16155 raft_consensus.cc:2454] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 1.
I20260806 06:41:29.646868 16011 raft_consensus.cc:2454] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 1.
I20260806 06:41:29.646982 15801 leader_election.cc:302] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068e7ca7b8bc43ee9e4b27bec85b5acd, 6f8f52d781c0479281f820f6e0bb3c60; no voters:
I20260806 06:41:29.647066 16237 raft_consensus.cc:2781] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:29.647265 16237 raft_consensus.cc:686] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 LEADER]: Becoming Leader. State: Replica: 068e7ca7b8bc43ee9e4b27bec85b5acd, State: Running, Role: LEADER
I20260806 06:41:29.647363 16237 consensus_queue.cc:237] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } }
I20260806 06:41:29.648134 15708 catalog_manager.cc:5495] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd reported cstate change: term changed from 0 to 1, leader changed from <none> to 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65). New cstate: current_term: 1 leader_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:29.662086 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:29.659889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.14.51.65:44081)
0806 06:41:29.659947 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:29.659948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:29.659974 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:41:29.659997 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:29.659998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:29.660011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:29.660056 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:29.660187 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.660612 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.661304 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:29.661476 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:29.661522 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:29.661690 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:29.661797 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:29.661856 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:29.661925 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:41:29.661960 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:41:29.663831 16011 raft_consensus.cc:1261] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Refusing update from remote peer 068e7ca7b8bc43ee9e4b27bec85b5acd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:29.664125 16240 consensus_queue.cc:1035] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Connected to new peer: Peer: permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:29.665808 16155 raft_consensus.cc:1261] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Refusing update from remote peer 068e7ca7b8bc43ee9e4b27bec85b5acd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:29.666075 16237 consensus_queue.cc:1035] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:29.670349 15709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:29.670483 15709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:29.671341 15709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Sending DeleteTablet for 3 replicas of tablet 2f2e13cfc6924adfac0b72e97bf40b9d
I20260806 06:41:29.671777 15849 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:29 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:43172
I20260806 06:41:29.671778 16134 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:29 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:42194
I20260806 06:41:29.672012 15991 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2e13cfc6924adfac0b72e97bf40b9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:29 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:60400
I20260806 06:41:29.671990 15709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:29.672134 15709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:29.672264 16254 tablet_replica.cc:330] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: stopping tablet replica
I20260806 06:41:29.672277 16255 tablet_replica.cc:330] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: stopping tablet replica
I20260806 06:41:29.672363 16255 raft_consensus.cc:2229] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:29.672485 16255 raft_consensus.cc:2258] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:29.672490 16254 raft_consensus.cc:2229] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:29.672644 16254 raft_consensus.cc:2258] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:29.673285 16253 tablet_replica.cc:330] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: stopping tablet replica
I20260806 06:41:29.673297 16255 ts_tablet_manager.cc:1905] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:29.673396 16253 raft_consensus.cc:2229] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:29.673462 16134 tablet_service.cc:1467] Processing CreateTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 (DEFAULT_TABLE table=test-table [id=48cbf00594cf4822af99feb5a42a0679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.673513 16253 raft_consensus.cc:2258] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:29.673563 16134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb8fecb9a0426f83a5a11df8361a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.673959 16253 ts_tablet_manager.cc:1905] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:29.675064 16228 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Bootstrap starting.
I20260806 06:41:29.675366 15849 tablet_service.cc:1467] Processing CreateTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 (DEFAULT_TABLE table=test-table [id=48cbf00594cf4822af99feb5a42a0679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.675467 15849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb8fecb9a0426f83a5a11df8361a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.675900 16228 tablet_bootstrap.cc:654] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.675984 16255 ts_tablet_manager.cc:1918] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:29.676052 16255 log.cc:1196] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/2f2e13cfc6924adfac0b72e97bf40b9d
I20260806 06:41:29.676678 16255 ts_tablet_manager.cc:1939] T 2f2e13cfc6924adfac0b72e97bf40b9d P 640ec919a3b84cb5be22db15f108f0f5: Deleting consensus metadata
I20260806 06:41:29.676898 16232 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Bootstrap starting.
I20260806 06:41:29.677343 16253 ts_tablet_manager.cc:1918] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:29.677415 16253 log.cc:1196] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/2f2e13cfc6924adfac0b72e97bf40b9d
I20260806 06:41:29.677397 15991 tablet_service.cc:1467] Processing CreateTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 (DEFAULT_TABLE table=test-table [id=48cbf00594cf4822af99feb5a42a0679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:29.677461 16232 tablet_bootstrap.cc:654] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.677495 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb8fecb9a0426f83a5a11df8361a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:29.677722 16253 ts_tablet_manager.cc:1939] T 2f2e13cfc6924adfac0b72e97bf40b9d P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting consensus metadata
I20260806 06:41:29.678480 16228 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: No bootstrap required, opened a new log
I20260806 06:41:29.678520 16228 ts_tablet_manager.cc:1397] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:41:29.678675 16228 raft_consensus.cc:348] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.678750 16228 raft_consensus.cc:374] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.678771 16228 raft_consensus.cc:729] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8f52d781c0479281f820f6e0bb3c60, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.678812 16228 consensus_queue.cc:260] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.678889 16228 ts_tablet_manager.cc:1428] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:29.679107 16232 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: No bootstrap required, opened a new log
I20260806 06:41:29.679169 16232 ts_tablet_manager.cc:1397] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 06:41:29.679240 16229 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Bootstrap starting.
I20260806 06:41:29.679466 16232 raft_consensus.cc:348] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.679569 16232 raft_consensus.cc:374] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.679598 16232 raft_consensus.cc:729] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068e7ca7b8bc43ee9e4b27bec85b5acd, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.679757 16232 consensus_queue.cc:260] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.679875 16232 ts_tablet_manager.cc:1428] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:29.679874 16229 tablet_bootstrap.cc:654] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:29.680562 15695 catalog_manager.cc:4841] TS 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961): tablet 2f2e13cfc6924adfac0b72e97bf40b9d (table test-table [id=aff0a47c0b1546dc8eff19e74357bddd]) successfully deleted
I20260806 06:41:29.680711 15695 catalog_manager.cc:4841] TS 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65:44081): tablet 2f2e13cfc6924adfac0b72e97bf40b9d (table test-table [id=aff0a47c0b1546dc8eff19e74357bddd]) successfully deleted
I20260806 06:41:29.681058 16229 tablet_bootstrap.cc:492] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: No bootstrap required, opened a new log
I20260806 06:41:29.681109 16229 ts_tablet_manager.cc:1397] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 06:41:29.681330 16229 raft_consensus.cc:348] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.681416 16229 raft_consensus.cc:374] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:29.681447 16229 raft_consensus.cc:729] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640ec919a3b84cb5be22db15f108f0f5, State: Initialized, Role: FOLLOWER
I20260806 06:41:29.681488 16229 consensus_queue.cc:260] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.681571 16229 ts_tablet_manager.cc:1428] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:29.681794 16254 ts_tablet_manager.cc:1905] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:29.683666 16254 ts_tablet_manager.cc:1918] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:29.683746 16254 log.cc:1196] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/2f2e13cfc6924adfac0b72e97bf40b9d
I20260806 06:41:29.684023 16254 ts_tablet_manager.cc:1939] T 2f2e13cfc6924adfac0b72e97bf40b9d P 6f8f52d781c0479281f820f6e0bb3c60: Deleting consensus metadata
I20260806 06:41:29.684401 15695 catalog_manager.cc:4841] TS 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765): tablet 2f2e13cfc6924adfac0b72e97bf40b9d (table test-table [id=aff0a47c0b1546dc8eff19e74357bddd]) successfully deleted
W20260806 06:41:29.794502 16206 tablet.cc:2369] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:29.970534 16237 raft_consensus.cc:482] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:29.970643 16237 raft_consensus.cc:504] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.970768 16237 leader_election.cc:288] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961), 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765)
I20260806 06:41:29.971002 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb8fecb9a0426f83a5a11df8361a13" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640ec919a3b84cb5be22db15f108f0f5" is_pre_election: true
I20260806 06:41:29.971035 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb8fecb9a0426f83a5a11df8361a13" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60" is_pre_election: true
I20260806 06:41:29.971143 16155 raft_consensus.cc:2454] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 0.
I20260806 06:41:29.971139 16011 raft_consensus.cc:2454] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 0.
I20260806 06:41:29.971282 15801 leader_election.cc:302] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068e7ca7b8bc43ee9e4b27bec85b5acd, 6f8f52d781c0479281f820f6e0bb3c60; no voters:
I20260806 06:41:29.971431 16237 raft_consensus.cc:2781] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:29.971486 16237 raft_consensus.cc:482] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:29.971525 16237 raft_consensus.cc:3037] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.972307 16237 raft_consensus.cc:504] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.972437 16237 leader_election.cc:288] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 election: Requested vote from peers 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961), 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765)
I20260806 06:41:29.972623 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb8fecb9a0426f83a5a11df8361a13" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640ec919a3b84cb5be22db15f108f0f5"
I20260806 06:41:29.972649 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb8fecb9a0426f83a5a11df8361a13" candidate_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60"
I20260806 06:41:29.972723 16011 raft_consensus.cc:3037] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.972749 16155 raft_consensus.cc:3037] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:29.973381 16155 raft_consensus.cc:2454] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 1.
I20260806 06:41:29.973382 16011 raft_consensus.cc:2454] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068e7ca7b8bc43ee9e4b27bec85b5acd in term 1.
I20260806 06:41:29.973560 15801 leader_election.cc:302] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068e7ca7b8bc43ee9e4b27bec85b5acd, 6f8f52d781c0479281f820f6e0bb3c60; no voters:
I20260806 06:41:29.973783 16237 raft_consensus.cc:2781] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:29.973847 16237 raft_consensus.cc:686] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 LEADER]: Becoming Leader. State: Replica: 068e7ca7b8bc43ee9e4b27bec85b5acd, State: Running, Role: LEADER
I20260806 06:41:29.973985 16237 consensus_queue.cc:237] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:29.974776 15709 catalog_manager.cc:5495] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd reported cstate change: term changed from 0 to 1, leader changed from <none> to 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65). New cstate: current_term: 1 leader_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:41:30 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998488, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:30.043149 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.035899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37539 (local address 127.0.0.1:39712)
0806 06:41:30.036141 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.036152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.036243 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.036374 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.036375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.036702 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:30.036762 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.036764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.037523 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.037524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.038104 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.038106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.038461 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.038462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.038514 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.038986 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:30.038991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:30.040562 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:30.041992 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:30.042000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:30.042002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:30.042196 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:30.042396 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:30.042397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:30.042397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:30.042527 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:30.042766 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:30.042786 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:30.042825 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.042903 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.043015 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":5,"thread_start_us":72,"threads_started":1}
I20260806 06:41:30.043149 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.035945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.14.51.126:37539)
0806 06:41:30.036105 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.036107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.036224 (+ 117us) server_negotiation.cc:407] Connection header received
0806 06:41:30.036315 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.036316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.036326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.036371 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:30.036890 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.037435 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.038183 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.038365 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.038460 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.040647 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:30.040648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:30.040649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:30.040655 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:30.041896 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.042271 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.042272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.042273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.042331 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.042594 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.042594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.042595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.042697 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:30.042764 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.042925 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.043060 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":135}
I20260806 06:41:30.045197 15709 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:39712:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:41:30.045454 15709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:30.046912 15849 tablet_service.cc:1467] Processing CreateTablet for tablet 03dcaf5d30aa4bcab979944f0794cb96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df6385b951ee421d9d2a4c0852933394]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:30.046981 16134 tablet_service.cc:1467] Processing CreateTablet for tablet 03dcaf5d30aa4bcab979944f0794cb96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df6385b951ee421d9d2a4c0852933394]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:30.047114 15849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dcaf5d30aa4bcab979944f0794cb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:30.047200 16134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dcaf5d30aa4bcab979944f0794cb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:30.048695 16232 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Bootstrap starting.
I20260806 06:41:30.048739 16228 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: Bootstrap starting.
I20260806 06:41:30.048862 15991 tablet_service.cc:1467] Processing CreateTablet for tablet 03dcaf5d30aa4bcab979944f0794cb96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df6385b951ee421d9d2a4c0852933394]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:30.048990 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dcaf5d30aa4bcab979944f0794cb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:30.049055 16232 tablet_bootstrap.cc:654] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:30.049139 16228 tablet_bootstrap.cc:654] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:30.049829 16232 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: No bootstrap required, opened a new log
I20260806 06:41:30.049883 16232 ts_tablet_manager.cc:1397] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:30.049998 16232 raft_consensus.cc:348] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.050084 16232 raft_consensus.cc:374] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:30.050105 16232 raft_consensus.cc:729] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068e7ca7b8bc43ee9e4b27bec85b5acd, State: Initialized, Role: FOLLOWER
I20260806 06:41:30.050150 16232 consensus_queue.cc:260] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.050199 16228 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: No bootstrap required, opened a new log
I20260806 06:41:30.050235 16232 ts_tablet_manager.cc:1428] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:30.050247 16228 ts_tablet_manager.cc:1397] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:30.050253 16237 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.050390 16237 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.050599 16228 raft_consensus.cc:348] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.050670 16228 raft_consensus.cc:374] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:30.050695 16228 raft_consensus.cc:729] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8f52d781c0479281f820f6e0bb3c60, State: Initialized, Role: FOLLOWER
I20260806 06:41:30.050742 16228 consensus_queue.cc:260] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.050841 16228 ts_tablet_manager.cc:1428] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:30.050853 16229 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: Bootstrap starting.
I20260806 06:41:30.051168 16233 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.051280 16233 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.051469 16229 tablet_bootstrap.cc:654] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:30.052265 16229 tablet_bootstrap.cc:492] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: No bootstrap required, opened a new log
I20260806 06:41:30.052325 16229 ts_tablet_manager.cc:1397] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:30.052634 16229 raft_consensus.cc:348] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.052711 16229 raft_consensus.cc:374] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:30.052735 16229 raft_consensus.cc:729] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640ec919a3b84cb5be22db15f108f0f5, State: Initialized, Role: FOLLOWER
I20260806 06:41:30.052805 16229 consensus_queue.cc:260] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.053006 16229 ts_tablet_manager.cc:1428] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:30.053063 16236 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.053170 16236 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.181977 16236 raft_consensus.cc:482] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:30.182070 16236 raft_consensus.cc:504] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.182377 16236 leader_election.cc:288] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65:44081), 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765)
I20260806 06:41:30.186458 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.182615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:46989 (local address 127.14.51.65:44081)
0806 06:41:30.182723 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.182725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.182750 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:41:30.182853 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.182855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.182868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.182936 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:30.183107 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.183916 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.185453 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.186034 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.186114 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.186127 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.186339 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.186387 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":44,"threads_started":1}
I20260806 06:41:30.186458 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.182801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:37437 (local address 127.14.51.67:38765)
0806 06:41:30.182911 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.182913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.182920 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:30.182944 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.182945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.182954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.183020 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:30.183182 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.183933 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.185454 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.186027 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.186156 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.186165 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.186339 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.186382 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260806 06:41:30.186537 16271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.182533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44081 (local address 127.14.51.66:46989)
0806 06:41:30.182667 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.182679 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.182753 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.182943 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.182943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.182945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:30.183034 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.183036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.184017 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.184018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.185363 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.185367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.186117 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.186119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.186263 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.186267 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.186319 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.186385 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":42,"thread_start_us":78,"threads_started":1}
I20260806 06:41:30.186549 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.182718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:38765 (local address 127.14.51.66:37437)
0806 06:41:30.182804 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.182812 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.182851 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.183032 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.183033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.183035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:30.183087 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.183089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.184017 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.184018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.185364 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.185367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.186163 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.186163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.186242 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.186257 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.186319 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.186385 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":62,"thread_start_us":35,"threads_started":1}
I20260806 06:41:30.186833 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dcaf5d30aa4bcab979944f0794cb96" candidate_uuid: "640ec919a3b84cb5be22db15f108f0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60" is_pre_election: true
I20260806 06:41:30.186941 16155 raft_consensus.cc:2454] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 640ec919a3b84cb5be22db15f108f0f5 in term 0.
I20260806 06:41:30.186873 15863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dcaf5d30aa4bcab979944f0794cb96" candidate_uuid: "640ec919a3b84cb5be22db15f108f0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" is_pre_election: true
I20260806 06:41:30.187016 15863 raft_consensus.cc:2454] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 640ec919a3b84cb5be22db15f108f0f5 in term 0.
I20260806 06:41:30.187166 15944 leader_election.cc:302] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 640ec919a3b84cb5be22db15f108f0f5, 6f8f52d781c0479281f820f6e0bb3c60; no voters:
I20260806 06:41:30.187314 16236 raft_consensus.cc:2781] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:30.187355 16236 raft_consensus.cc:482] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:30.187395 16236 raft_consensus.cc:3037] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:30.188287 16236 raft_consensus.cc:504] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.188416 16236 leader_election.cc:288] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65:44081), 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765)
I20260806 06:41:30.188611 15863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dcaf5d30aa4bcab979944f0794cb96" candidate_uuid: "640ec919a3b84cb5be22db15f108f0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd"
I20260806 06:41:30.188659 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dcaf5d30aa4bcab979944f0794cb96" candidate_uuid: "640ec919a3b84cb5be22db15f108f0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8f52d781c0479281f820f6e0bb3c60"
I20260806 06:41:30.188709 15863 raft_consensus.cc:3037] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:30.188731 16155 raft_consensus.cc:3037] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:30.189337 16155 raft_consensus.cc:2454] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 640ec919a3b84cb5be22db15f108f0f5 in term 1.
I20260806 06:41:30.189370 15863 raft_consensus.cc:2454] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 640ec919a3b84cb5be22db15f108f0f5 in term 1.
I20260806 06:41:30.189524 15944 leader_election.cc:302] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068e7ca7b8bc43ee9e4b27bec85b5acd, 640ec919a3b84cb5be22db15f108f0f5; no voters:
I20260806 06:41:30.189643 16236 raft_consensus.cc:2781] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:30.189765 16236 raft_consensus.cc:686] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 LEADER]: Becoming Leader. State: Replica: 640ec919a3b84cb5be22db15f108f0f5, State: Running, Role: LEADER
I20260806 06:41:30.189849 16236 consensus_queue.cc:237] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } }
I20260806 06:41:30.190171 16275 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: TxnStatusTablet state changed. Reason: New leader 640ec919a3b84cb5be22db15f108f0f5. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.190274 16275 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:30.190538 16278 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:41:30.190526 15708 catalog_manager.cc:5495] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:30.190634 16278 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:41:30.196527 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.193746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45961 (local address 127.0.0.1:60408)
0806 06:41:30.193818 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.193834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.193869 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.194123 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.194124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.194125 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:30.194173 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.194176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.194881 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.194882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.195539 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.195541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.195980 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.195981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.196039 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.196055 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.196337 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.196348 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.196393 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.196442 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:41:30.196523 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.193800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.14.51.66:45961)
0806 06:41:30.193983 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.193985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.193991 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:41:30.194015 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.194015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.194029 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.194103 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:30.194225 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.194792 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.195623 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.195900 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.195978 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.196155 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.196269 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.196351 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.196396 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.196415 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260806 06:41:30.197783 16155 raft_consensus.cc:1261] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Refusing update from remote peer 640ec919a3b84cb5be22db15f108f0f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:30.197854 15863 raft_consensus.cc:1261] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Refusing update from remote peer 640ec919a3b84cb5be22db15f108f0f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:30.198129 16236 consensus_queue.cc:1035] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:30.198230 16275 consensus_queue.cc:1035] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:30.198770 16233 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: TxnStatusTablet state changed. Reason: New leader 640ec919a3b84cb5be22db15f108f0f5. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.198850 16233 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.199117 16237 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: TxnStatusTablet state changed. Reason: New leader 640ec919a3b84cb5be22db15f108f0f5. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.199223 16237 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.199548 16236 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.199698 16236 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:30.200187 16282 mvcc.cc:205] Tried to move back new op lower bound from 7315449815848140800 to 7315449815818153984. Current Snapshot: MvccSnapshot[applied={T|T < 7315449815848140800 or (T in {7315449815848140800})}]
I20260806 06:41:30.200269 16285 mvcc.cc:205] Tried to move back new op lower bound from 7315449815848140800 to 7315449815818153984. Current Snapshot: MvccSnapshot[applied={T|T < 7315449815848140800}]
I20260806 06:41:30.200470 16237 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.200551 16237 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 068e7ca7b8bc43ee9e4b27bec85b5acd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.201712 16275 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.201835 16275 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 640ec919a3b84cb5be22db15f108f0f5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:30.202252 16233 tablet_replica.cc:439] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "640ec919a3b84cb5be22db15f108f0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068e7ca7b8bc43ee9e4b27bec85b5acd" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44081 } } peers { permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 } } peers { permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 } } }
I20260806 06:41:30.202359 16233 tablet_replica.cc:442] T 03dcaf5d30aa4bcab979944f0794cb96 P 6f8f52d781c0479281f820f6e0bb3c60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:30.202621 16283 mvcc.cc:205] Tried to move back new op lower bound from 7315449815848140800 to 7315449815818153984. Current Snapshot: MvccSnapshot[applied={T|T < 7315449815848140800 or (T in {7315449815848140800})}]
I20260806 06:41:30.206909 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.203957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45961 (local address 127.14.51.65:58791)
0806 06:41:30.204196 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.204209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.204266 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.204419 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.204420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.204421 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:30.204457 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.204458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.205392 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.205393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.206034 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.206037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.206522 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.206523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.206589 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.206609 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.206783 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.206785 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.206817 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.206856 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":149,"threads_started":1}
I20260806 06:41:30.206914 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.204032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:58791 (local address 127.14.51.66:45961)
0806 06:41:30.204079 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.204081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.204312 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:41:30.204337 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.204338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.204349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.204415 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:30.204496 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.205268 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.206122 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.206425 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.206497 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.206691 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.206723 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.206775 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.206827 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.206848 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:41:30.209125 16011 raft_consensus.cc:1261] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Refusing update from remote peer 068e7ca7b8bc43ee9e4b27bec85b5acd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:30.209151 16155 raft_consensus.cc:1261] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Refusing update from remote peer 068e7ca7b8bc43ee9e4b27bec85b5acd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:30.209314 16246 consensus_queue.cc:1035] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Connected to new peer: Peer: permanent_uuid: "640ec919a3b84cb5be22db15f108f0f5" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:30.209436 16237 consensus_queue.cc:1035] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8f52d781c0479281f820f6e0bb3c60" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:30.218353 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.215522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:53401 (local address 127.14.51.65:44081)
0806 06:41:30.215669 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.215671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.215684 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:41:30.215753 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.215754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.215766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.215821 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:30.215998 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.216444 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.217734 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.217927 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.217981 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.218182 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.218206 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.218238 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.218291 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.218307 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260806 06:41:30.218431 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.215449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44081 (local address 127.14.51.66:53401)
0806 06:41:30.215642 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:30.215654 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:30.215700 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:30.215862 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:30.215863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:30.215865 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:30.215929 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.215930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.216542 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.216544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.217622 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:30.217625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.217997 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.217998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:30.218083 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.218097 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:30.218240 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:30.218241 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:30.218289 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:41:30.218328 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260806 06:41:30.226994 15708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:30.227100 15708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:30.227658 15708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bb0d8d0f9724b23a7aa8369b0b3be42: Sending DeleteTablet for 3 replicas of tablet bccb8fecb9a0426f83a5a11df8361a13
I20260806 06:41:30.227944 15849 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:30 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:43172
I20260806 06:41:30.228030 16134 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:30 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:42194
I20260806 06:41:30.228070 15991 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb8fecb9a0426f83a5a11df8361a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:30 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:60400
I20260806 06:41:30.228328 16301 tablet_replica.cc:330] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: stopping tablet replica
I20260806 06:41:30.228399 16301 raft_consensus.cc:2229] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:30.228430 16301 raft_consensus.cc:2258] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:30.228626 16300 tablet_replica.cc:330] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: stopping tablet replica
I20260806 06:41:30.228704 16300 raft_consensus.cc:2229] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:30.228768 16300 raft_consensus.cc:2258] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:30.228878 16301 ts_tablet_manager.cc:1905] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:30.228967 16300 ts_tablet_manager.cc:1905] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:30.230985 16300 ts_tablet_manager.cc:1918] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:30.231055 16300 log.cc:1196] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/bccb8fecb9a0426f83a5a11df8361a13
I20260806 06:41:30.231403 16300 ts_tablet_manager.cc:1939] T bccb8fecb9a0426f83a5a11df8361a13 P 068e7ca7b8bc43ee9e4b27bec85b5acd: Deleting consensus metadata
I20260806 06:41:30.232210 15695 catalog_manager.cc:4841] TS 068e7ca7b8bc43ee9e4b27bec85b5acd (127.14.51.65:44081): tablet bccb8fecb9a0426f83a5a11df8361a13 (table test-table [id=48cbf00594cf4822af99feb5a42a0679]) successfully deleted
Aug 06 06:41:30 dist-test-slave-1xrh krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998489, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:30.235217 16301 ts_tablet_manager.cc:1918] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:30.235266 16301 log.cc:1196] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/bccb8fecb9a0426f83a5a11df8361a13
I20260806 06:41:30.235389 16302 tablet_replica.cc:330] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: stopping tablet replica
I20260806 06:41:30.235522 16301 ts_tablet_manager.cc:1939] T bccb8fecb9a0426f83a5a11df8361a13 P 6f8f52d781c0479281f820f6e0bb3c60: Deleting consensus metadata
I20260806 06:41:30.235459 16302 raft_consensus.cc:2229] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:30.235847 16302 raft_consensus.cc:2258] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:30.235852 15695 catalog_manager.cc:4841] TS 6f8f52d781c0479281f820f6e0bb3c60 (127.14.51.67:38765): tablet bccb8fecb9a0426f83a5a11df8361a13 (table test-table [id=48cbf00594cf4822af99feb5a42a0679]) successfully deleted
I20260806 06:41:30.236112 16302 ts_tablet_manager.cc:1905] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:30.238150 16302 ts_tablet_manager.cc:1918] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:30.238222 16302 log.cc:1196] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/bccb8fecb9a0426f83a5a11df8361a13
I20260806 06:41:30.238480 16302 ts_tablet_manager.cc:1939] T bccb8fecb9a0426f83a5a11df8361a13 P 640ec919a3b84cb5be22db15f108f0f5: Deleting consensus metadata
I20260806 06:41:30.239034 15695 catalog_manager.cc:4841] TS 640ec919a3b84cb5be22db15f108f0f5 (127.14.51.66:45961): tablet bccb8fecb9a0426f83a5a11df8361a13 (table test-table [id=48cbf00594cf4822af99feb5a42a0679]) successfully deleted
I20260806 06:41:30.239472 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.228697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.14.51.65:44081)
0806 06:41:30.228730 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.228732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.230858 (+ 2126us) server_negotiation.cc:407] Connection header received
0806 06:41:30.230914 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.230915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.230925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.230972 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:30.231611 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.232035 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.232856 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.233070 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.233144 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.235782 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:30.235793 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:30.235794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:30.235802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:30.237852 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.238325 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.238328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.238329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.238391 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.238762 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.238764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.238764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.238927 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:30.239066 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.239282 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.239374 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:41:30.239744 15909 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43190
I20260806 06:41:30.247382 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:30.240732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.14.51.126:37539)
0806 06:41:30.240780 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:30.240782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:30.240876 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:41:30.240899 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:30.240900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:30.240909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:30.240961 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:30.241478 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.241993 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.242820 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:30.243047 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:30.243128 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:30.244421 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:30.244423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:30.244423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:30.244431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:30.245973 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.246382 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.246383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.246384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.246447 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:30.246721 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:30.246722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:30.246723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:30.246937 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:30.247041 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:30.247206 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:41:30.247283 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:41:30.247680 15708 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39726
I20260806 06:41:30.248159 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15783
I20260806 06:41:30.257107 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15926
I20260806 06:41:30.263023 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 16068
I20260806 06:41:30.268162 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 15676
2026-08-06T06:41:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3374 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:30 dist-test-slave-1xrh krb5kdc[16309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:30 dist-test-slave-1xrh krb5kdc[16309](info): set up 2 sockets
Aug 06 06:41:30 dist-test-slave-1xrh krb5kdc[16309](info): commencing operation
krb5kdc: starting...
W20260806 06:41:31.316460 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:31.434341 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40339
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:40339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:31.516619 16322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:31.516829 16322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:31.516852 16322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:31.518013 16322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:31.518054 16322 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:41:31.518069 16322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:31.518081 16322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:31.518093 16322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:31.518105 16322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:31.519574 16322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:40339
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:31.519833 16322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:31.520071 16322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:31.525087 16328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:31.525118 16327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.525216 16322 server_base.cc:1034] running on GCE node
W20260806 06:41:31.525269 16330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.525563 16322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:31.525848 16322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:31.526991 16322 hybrid_clock.cc:648] HybridClock initialized: now 1785998491526958 us; error 53 us; skew 500 ppm
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:31.529381 16322 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:31.529758 16322 webserver.cc:460] Webserver started at http://127.14.51.126:40455/ using document root <none> and password file <none>
I20260806 06:41:31.529934 16322 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:31.529974 16322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:31.530068 16322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:31.531061 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "4fdebf5ab8794689bcb706621ae1aad4"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "31c9de35e724c59cebe929bf9b81d38d"
server_key_iv: "2efc408c2aa36f90be2b3a3ac17c1c02"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.531458 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "4fdebf5ab8794689bcb706621ae1aad4"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "31c9de35e724c59cebe929bf9b81d38d"
server_key_iv: "2efc408c2aa36f90be2b3a3ac17c1c02"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.532698 16322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:31.533380 16340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.533548 16322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:31.533619 16322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "4fdebf5ab8794689bcb706621ae1aad4"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "31c9de35e724c59cebe929bf9b81d38d"
server_key_iv: "2efc408c2aa36f90be2b3a3ac17c1c02"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.533670 16322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:31.541319 16322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:31.543524 16322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:31.543628 16322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:31.547960 16322 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:40339
I20260806 06:41:31.548043 16403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:40339 every 8 connection(s)
I20260806 06:41:31.548344 16322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:31.548651 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 16322
I20260806 06:41:31.548743 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:31.548911 14541 external_mini_cluster.cc:1428] Setting key 1be3f41fcd0eefb6c1c30395b1abf9a7
I20260806 06:41:31.549068 16404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:31.552322 16404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Bootstrap starting.
I20260806 06:41:31.553007 16404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:31.553301 16404 log.cc:824] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:31.554055 16404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: No bootstrap required, opened a new log
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:31.556017 16404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } }
I20260806 06:41:31.556166 16404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:31.556216 16404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fdebf5ab8794689bcb706621ae1aad4, State: Initialized, Role: FOLLOWER
I20260806 06:41:31.556357 16404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } }
I20260806 06:41:31.556419 16404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:31.556468 16404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:31.556517 16404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:31.557036 16404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } }
I20260806 06:41:31.557120 16404 leader_election.cc:302] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fdebf5ab8794689bcb706621ae1aad4; no voters:
I20260806 06:41:31.557300 16404 leader_election.cc:288] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:31.557353 16410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:31.557472 16410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [term 1 LEADER]: Becoming Leader. State: Replica: 4fdebf5ab8794689bcb706621ae1aad4, State: Running, Role: LEADER
I20260806 06:41:31.557556 16404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:31.557549 16410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } }
I20260806 06:41:31.558015 16410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fdebf5ab8794689bcb706621ae1aad4. Latest consensus state: current_term: 1 leader_uuid: "4fdebf5ab8794689bcb706621ae1aad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } } }
I20260806 06:41:31.558260 16410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:31.558184 16412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fdebf5ab8794689bcb706621ae1aad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdebf5ab8794689bcb706621ae1aad4" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40339 } } }
I20260806 06:41:31.558523 16412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:31.558914 16419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:31.559509 16419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:31.560266 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.549440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.14.51.126:40339)
0806 06:41:31.549612 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:31.549620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:31.549655 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:41:31.550162 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:31.550168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:31.550211 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:31.550319 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:31.551089 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.552009 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.553076 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.553519 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.553636 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.555956 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:31.555960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:31.555960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:31.555972 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:31.558102 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.558603 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.558605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.558606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.558706 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.558945 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.558946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.558947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.559309 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:31.559893 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:31.559963 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:41:31.560099 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260806 06:41:31.561257 16419 catalog_manager.cc:1347] Generated new cluster ID: 69cce9d0a3304d32a5253a834bb2850a
I20260806 06:41:31.561308 16419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:31.571961 16419 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:31.572602 16419 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:31.576427 16419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Generated new TSK 0
I20260806 06:41:31.576604 16419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:31.606150 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40339
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:31.687470 16431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:31.687690 16431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:31.687732 16431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:31.688905 16431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:31.688966 16431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:31.689013 16431 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:41:31.689059 16431 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:31.689100 16431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:31.690546 16431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:31.690807 16431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:31.691043 16431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:31.696041 16436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:31.696044 16437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.696063 16431 server_base.cc:1034] running on GCE node
W20260806 06:41:31.696044 16439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.696493 16431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:31.696688 16431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:31.697822 16431 hybrid_clock.cc:648] HybridClock initialized: now 1785998491697790 us; error 49 us; skew 500 ppm
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:31.700338 16431 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:31.700781 16431 webserver.cc:460] Webserver started at http://127.14.51.65:34963/ using document root <none> and password file <none>
I20260806 06:41:31.700973 16431 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:31.701020 16431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:31.701105 16431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:31.702062 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "9ceafbce626f48cba7daa5b0c9089fea"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "442af77f389cb062f65ee711816672ef"
server_key_iv: "f694dc830cd8a056b5e7d3b4f27d84b1"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.702358 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "9ceafbce626f48cba7daa5b0c9089fea"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "442af77f389cb062f65ee711816672ef"
server_key_iv: "f694dc830cd8a056b5e7d3b4f27d84b1"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.703691 16431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:31.704385 16446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.704561 16431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:31.704659 16431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "9ceafbce626f48cba7daa5b0c9089fea"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "442af77f389cb062f65ee711816672ef"
server_key_iv: "f694dc830cd8a056b5e7d3b4f27d84b1"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.704736 16431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:31.727727 16431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:31.729557 16431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:31.729714 16431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:31.730304 16431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:31.730356 16431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.730391 16431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:31.730412 16431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:31.737829 16431 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:35343
I20260806 06:41:31.737898 16567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:35343 every 8 connection(s)
I20260806 06:41:31.738194 16431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:31.740828 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.730712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:33805 (local address 127.14.51.126:40339)
0806 06:41:31.730781 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:31.730783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:31.731030 (+ 247us) server_negotiation.cc:407] Connection header received
0806 06:41:31.731571 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:31.731573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:31.731587 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:31.731675 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:31.732205 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.733050 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.734103 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.734312 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.734390 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.737806 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:31.737808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:31.737808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:31.737817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:31.739432 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.739857 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.739858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.739859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.739918 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.740173 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.740174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.740175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.740295 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:31.740368 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:31.740656 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 06:41:31.740741 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:41:31.740999 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.730717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.65:33805)
0806 06:41:31.730890 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:31.730949 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:31.731473 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:31.731727 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:31.731730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:31.731955 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:31.732129 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.732133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.733190 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.733191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.733964 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.733967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.734406 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.734407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.734598 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.735170 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:31.735182 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:31.737710 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:31.739533 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.739539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.739547 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.739766 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.739991 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.739992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.739992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.740103 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.740371 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:31.740378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:31.740510 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:41:31.740636 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 06:41:31.740783 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":107,"threads_started":1}
I20260806 06:41:31.741221 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 16431
I20260806 06:41:31.741300 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:31.741431 14541 external_mini_cluster.cc:1428] Setting key 6e00dd5512b69a48dc74cd3bab4c58c5
I20260806 06:41:31.742003 16568 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40339
I20260806 06:41:31.742110 16568 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:31.742291 16568 heartbeater.cc:511] Master 127.14.51.126:40339 requested a full tablet report, sending...
I20260806 06:41:31.742803 16358 ts_manager.cc:194] Registered new tserver with Master: 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343)
I20260806 06:41:31.743726 16358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:33805
I20260806 06:41:31.749029 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.743254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:55065 (local address 127.14.51.126:40339)
0806 06:41:31.743290 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:31.743292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:31.743299 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:31.743319 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:31.743320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:31.743330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:31.743389 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:31.743751 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.744346 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.745196 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.745378 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.745436 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.746530 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:31.746531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:31.746532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:31.746537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:31.747900 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.748328 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.748328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.748329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.748375 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.748596 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.748597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.748597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.748685 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:31.748733 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:31.748913 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:41:31.748966 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:31.749119 16574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.743031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.65:55065)
0806 06:41:31.743197 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:31.743209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:31.743261 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:31.743414 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:31.743414 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:31.743600 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:31.743656 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.743658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.744458 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.744459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.745113 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.745116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.745456 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.745457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.745550 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.745923 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:31.745928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:31.746436 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:31.748004 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.748005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.748006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.748251 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.748449 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.748450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.748451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.748524 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.748751 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:31.748752 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:31.748802 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:41:31.748898 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:41:31.749007 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:31.769672 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40339
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:31.849735 16577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:31.849959 16577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:31.849998 16577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:31.851183 16577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:31.851251 16577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:31.851285 16577 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:41:31.851322 16577 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:31.851356 16577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:31.852751 16577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:31.853011 16577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:31.853241 16577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:31.858105 16583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:31.858150 16585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.858405 16577 server_base.cc:1034] running on GCE node
W20260806 06:41:31.858148 16582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:31.858669 16577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:31.858914 16577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:31.860049 16577 hybrid_clock.cc:648] HybridClock initialized: now 1785998491860044 us; error 31 us; skew 500 ppm
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:31.862730 16577 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:31.863240 16577 webserver.cc:460] Webserver started at http://127.14.51.66:43795/ using document root <none> and password file <none>
I20260806 06:41:31.863468 16577 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:31.863508 16577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:31.863596 16577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:31.864584 16577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "f0364123266c450abbaa1fc323569a3c"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "5cd7c8066bd868b7a91ecb63fcdaadb7"
server_key_iv: "4d29d7b6d2d04ba82b6d3e4d68d3810b"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.864873 16577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "f0364123266c450abbaa1fc323569a3c"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "5cd7c8066bd868b7a91ecb63fcdaadb7"
server_key_iv: "4d29d7b6d2d04ba82b6d3e4d68d3810b"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.866078 16577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:31.866725 16592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.866901 16577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:31.866993 16577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "f0364123266c450abbaa1fc323569a3c"
format_stamp: "Formatted at 2026-08-06 06:41:31 on dist-test-slave-1xrh"
server_key: "5cd7c8066bd868b7a91ecb63fcdaadb7"
server_key_iv: "4d29d7b6d2d04ba82b6d3e4d68d3810b"
server_key_version: "encryptionkey@0"
I20260806 06:41:31.867063 16577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:31.879657 16577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:31.881831 16577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:31.881992 16577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:31.882478 16577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:31.882515 16577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.882536 16577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:31.882579 16577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:31.889242 16577 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:42521
I20260806 06:41:31.889364 16711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:42521 every 8 connection(s)
I20260806 06:41:31.889649 16577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:31 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:31.893405 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.883174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:57197 (local address 127.14.51.126:40339)
0806 06:41:31.883230 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:31.883232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:31.883455 (+ 223us) server_negotiation.cc:407] Connection header received
0806 06:41:31.883972 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:31.883973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:31.883987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:31.884067 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:31.884962 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.885806 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.887394 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.887662 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.887744 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.890288 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:31.890289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:31.890290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:31.890296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:31.891724 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.892228 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.892229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.892229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.892295 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.892738 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.892739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.892740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.892862 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:31.892937 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:31.893217 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 06:41:31.893305 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:31.893550 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.883091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.66:57197)
0806 06:41:31.883323 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:31.883360 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:31.883830 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:31.884162 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:31.884167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:31.884523 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:31.884851 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.884857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.885894 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.885894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.887308 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.887312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.887748 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.887750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.887878 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.888413 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:31.888424 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:31.890196 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:31.891835 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.891842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.891844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.892156 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.892550 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.892550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.892551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.892655 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.892929 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:31.892935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:31.893076 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:41:31.893180 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:41:31.893340 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":163,"threads_started":1}
I20260806 06:41:31.894378 16712 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40339
I20260806 06:41:31.894493 16712 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:31.894702 16712 heartbeater.cc:511] Master 127.14.51.126:40339 requested a full tablet report, sending...
I20260806 06:41:31.895107 16356 ts_manager.cc:194] Registered new tserver with Master: f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521)
I20260806 06:41:31.895355 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 16577
I20260806 06:41:31.895493 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:31.895651 16356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:57197
I20260806 06:41:31.895669 14541 external_mini_cluster.cc:1428] Setting key 76fde22c41f2429d8334e149d6f0879d
I20260806 06:41:31.901252 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.895454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44885 (local address 127.14.51.126:40339)
0806 06:41:31.895505 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:31.895507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:31.895701 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:41:31.895819 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:31.895819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:31.895830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:31.895878 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:31.896237 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.896750 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.897454 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.897657 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.897733 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.898632 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:31.898633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:31.898634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:31.898640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:31.900083 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.900438 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.900439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.900439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.900494 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:31.900705 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:31.900706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:31.900706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:31.900854 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:31.900929 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:31.901100 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:41:31.901168 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:41:31.901260 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:31.895397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.66:44885)
0806 06:41:31.895621 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:31.895631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:31.895695 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:31.895894 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:31.895894 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:31.896102 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:31.896157 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.896159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.896850 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.896851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.897352 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:31.897354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:31.897732 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:31.897733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:31.897811 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:31.898241 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:31.898246 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:31.898525 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:31.900185 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.900186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.900186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.900362 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.900571 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:31.900572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:31.900572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:31.900632 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:31.900933 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:31.900935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:31.900980 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:41:31.901060 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:41:31.901163 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:31.921438 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40339
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:32.001814 16721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:32.002036 16721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:32.002063 16721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:32.003271 16721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:32.003314 16721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:32.003335 16721 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:41:32.003376 16721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:32.003412 16721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:32.004846 16721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:32.005100 16721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:32.005354 16721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:32.010450 16726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:32.010478 16729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:32.010466 16727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:32.010537 16721 server_base.cc:1034] running on GCE node
I20260806 06:41:32.010839 16721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:32.011111 16721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:32.012300 16721 hybrid_clock.cc:648] HybridClock initialized: now 1785998492012290 us; error 31 us; skew 500 ppm
Aug 06 06:41:32 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998492, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:32.014765 16721 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:32.015318 16721 webserver.cc:460] Webserver started at http://127.14.51.67:32935/ using document root <none> and password file <none>
I20260806 06:41:32.015568 16721 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:32.015610 16721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:32.015698 16721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:32.016641 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "e5979826131144869ce09929102c9d6c"
format_stamp: "Formatted at 2026-08-06 06:41:32 on dist-test-slave-1xrh"
server_key: "9d8c0aa51b0d2029cbba70503c4d7ff6"
server_key_iv: "9851f5af8ad76cbd7f5664ad4945310a"
server_key_version: "encryptionkey@0"
I20260806 06:41:32.016959 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "e5979826131144869ce09929102c9d6c"
format_stamp: "Formatted at 2026-08-06 06:41:32 on dist-test-slave-1xrh"
server_key: "9d8c0aa51b0d2029cbba70503c4d7ff6"
server_key_iv: "9851f5af8ad76cbd7f5664ad4945310a"
server_key_version: "encryptionkey@0"
I20260806 06:41:32.018188 16721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:32.018833 16737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.019018 16721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.019093 16721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "e5979826131144869ce09929102c9d6c"
format_stamp: "Formatted at 2026-08-06 06:41:32 on dist-test-slave-1xrh"
server_key: "9d8c0aa51b0d2029cbba70503c4d7ff6"
server_key_iv: "9851f5af8ad76cbd7f5664ad4945310a"
server_key_version: "encryptionkey@0"
I20260806 06:41:32.019192 16721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:32.030910 16721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:32.032672 16721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:32.032796 16721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:32.033344 16721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:32.033404 16721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.033433 16721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:32.033452 16721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:32 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998492, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:32.041038 16721 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:41973
I20260806 06:41:32.041469 16859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:41973 every 8 connection(s)
I20260806 06:41:32.041513 16721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:32.043439 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.033757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:49729 (local address 127.14.51.126:40339)
0806 06:41:32.033842 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.033844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.034089 (+ 245us) server_negotiation.cc:407] Connection header received
0806 06:41:32.034792 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.034793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.034803 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.034847 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:32.035773 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.036380 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.037661 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.037860 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.037938 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.040628 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:32.040629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:32.040629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:32.040635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:32.042328 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.042722 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.042723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.042723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.042769 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.042917 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.042917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.042917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.043006 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:32.043048 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.043325 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.043381 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:41:32.043601 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.033736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.67:49729)
0806 06:41:32.033942 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.033994 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.034682 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.034921 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.034926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.035340 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:32.035677 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.035684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.036476 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.036477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.037572 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.037575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.037946 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.037947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.038076 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.038568 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:32.038577 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:32.040539 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:32.042433 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.042437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.042439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.042661 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.042807 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.042808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.042808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.042884 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.043056 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:32.043059 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:32.043207 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.043297 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.043425 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260806 06:41:32.044075 16860 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40339
I20260806 06:41:32.044158 16860 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:32.044329 16860 heartbeater.cc:511] Master 127.14.51.126:40339 requested a full tablet report, sending...
I20260806 06:41:32.044818 16356 ts_manager.cc:194] Registered new tserver with Master: e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:32.045274 16356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:49729
I20260806 06:41:32.045897 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 16721
I20260806 06:41:32.046002 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:32.046206 14541 external_mini_cluster.cc:1428] Setting key b7a6208f31270a03e1905a7a166755dc
I20260806 06:41:32.047359 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:41:32.051156 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.045106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:58403 (local address 127.14.51.126:40339)
0806 06:41:32.045141 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.045143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.045275 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:41:32.045369 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.045370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.045379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.045470 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:32.045789 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.046388 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.047327 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.047605 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.047661 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.048608 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:32.048609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:32.048609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:32.048614 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:32.049942 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.050313 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.050314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.050315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.050374 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.050592 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.050592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.050593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.050696 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:32.050766 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.050940 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.051022 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:32.051156 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.045039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.14.51.67:58403)
0806 06:41:32.045204 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.045212 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.045289 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.045490 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.045491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.045659 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:32.045712 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.045713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.046506 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.046508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.047248 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.047251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.047680 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.047681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.047767 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.048121 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:32.048126 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:32.048513 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:32.050059 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.050060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.050060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.050229 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.050446 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.050447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.050447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.050523 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.050771 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:32.050772 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:32.050813 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.050890 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.051010 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
Aug 06 06:41:32 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:41:32 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:32.069905 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.063727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.14.51.126:40339)
0806 06:41:32.063784 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.063785 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.063810 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:41:32.063830 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.063830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.063837 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.063904 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:32.064400 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.064830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.065489 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.065654 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.065700 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.067847 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:32.067848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:32.067848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:32.067854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:32.068937 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.069225 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.069226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.069226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.069270 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.069446 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.069447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.069447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.069548 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:32.069663 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.069795 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.069850 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:41:32.071504 16356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:32.073065 16356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:32.079512 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:35343 (local address 127.0.0.1:34098)
0806 06:41:32.075671 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.075683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.075726 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.076054 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.076055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.076057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.076122 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.076124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077154 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.077156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.078509 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.078511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079202 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.079293 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079308 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.079377 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.079431 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":41,"thread_start_us":92,"threads_started":1}
I20260806 06:41:32.079633 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.14.51.67:41973)
0806 06:41:32.076047 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.076052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.076061 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:32.076117 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.076119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.076134 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.076371 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.076422 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077424 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.078481 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079223 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079338 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079399 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.079495 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.079525 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":80,"threads_started":1}
I20260806 06:41:32.079979 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.14.51.66:42521)
0806 06:41:32.075745 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.075750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.075760 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:32.075798 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.075800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.075817 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.075927 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.076083 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077254 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.078544 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079286 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079351 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079399 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.079835 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.079880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":88,"threads_started":1}
I20260806 06:41:32.080008 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.14.51.65:35343)
0806 06:41:32.075878 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.075882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.075893 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:32.075929 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.075931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.075946 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.076044 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.076205 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077069 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.078594 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079077 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079201 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079265 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.079864 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.079903 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260806 06:41:32.079641 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:41973 (local address 127.0.0.1:38134)
0806 06:41:32.075541 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.075554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.075718 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.076236 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.076237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.076239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.076286 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.076287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077504 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.077505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.078397 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.078399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079343 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.079411 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079414 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.079468 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.079500 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":2,"thread_start_us":35,"threads_started":1}
I20260806 06:41:32.079761 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.075220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.0.0.1:48382)
0806 06:41:32.075376 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.075412 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.075588 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.075928 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.075931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.075933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.075994 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.075996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.077347 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.077348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.078469 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.078471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.079548 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.079548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.079615 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.079619 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.079670 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.079701 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260806 06:41:32.080788 16791 tablet_service.cc:1467] Processing CreateTablet for tablet be495c4fae3b44efa7d15af8ba69f317 (DEFAULT_TABLE table=test-table [id=550833730abd4bccb3fd233fde670b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.080788 16646 tablet_service.cc:1467] Processing CreateTablet for tablet be495c4fae3b44efa7d15af8ba69f317 (DEFAULT_TABLE table=test-table [id=550833730abd4bccb3fd233fde670b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.081133 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be495c4fae3b44efa7d15af8ba69f317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.081133 16791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be495c4fae3b44efa7d15af8ba69f317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.081126 16500 tablet_service.cc:1467] Processing CreateTablet for tablet be495c4fae3b44efa7d15af8ba69f317 (DEFAULT_TABLE table=test-table [id=550833730abd4bccb3fd233fde670b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.081473 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be495c4fae3b44efa7d15af8ba69f317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.084393 16880 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Bootstrap starting.
I20260806 06:41:32.084642 16881 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Bootstrap starting.
I20260806 06:41:32.084816 16882 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Bootstrap starting.
I20260806 06:41:32.085294 16880 tablet_bootstrap.cc:654] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.085434 16881 tablet_bootstrap.cc:654] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.085441 16882 tablet_bootstrap.cc:654] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.085656 16880 log.cc:824] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:32.085744 16881 log.cc:824] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:32.085753 16882 log.cc:824] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:32.086436 16882 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: No bootstrap required, opened a new log
I20260806 06:41:32.086458 16880 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: No bootstrap required, opened a new log
I20260806 06:41:32.086511 16882 ts_tablet_manager.cc:1397] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:32.086536 16880 ts_tablet_manager.cc:1397] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:32.086671 16881 tablet_bootstrap.cc:492] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: No bootstrap required, opened a new log
I20260806 06:41:32.086720 16881 ts_tablet_manager.cc:1397] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:32.088071 16880 raft_consensus.cc:348] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.088227 16880 raft_consensus.cc:374] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.088271 16880 raft_consensus.cc:729] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceafbce626f48cba7daa5b0c9089fea, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.088389 16880 consensus_queue.cc:260] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.088528 16882 raft_consensus.cc:348] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.088681 16882 raft_consensus.cc:374] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.088678 16880 ts_tablet_manager.cc:1428] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:41:32.088716 16882 raft_consensus.cc:729] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.088688 16568 heartbeater.cc:503] Master 127.14.51.126:40339 was elected leader, sending a full tablet report...
I20260806 06:41:32.088761 16881 raft_consensus.cc:348] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.088855 16882 consensus_queue.cc:260] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.088909 16881 raft_consensus.cc:374] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.088955 16881 raft_consensus.cc:729] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5979826131144869ce09929102c9d6c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.089072 16881 consensus_queue.cc:260] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.089145 16712 heartbeater.cc:503] Master 127.14.51.126:40339 was elected leader, sending a full tablet report...
I20260806 06:41:32.089131 16882 ts_tablet_manager.cc:1428] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:32.089303 16881 ts_tablet_manager.cc:1428] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:32.089461 16860 heartbeater.cc:503] Master 127.14.51.126:40339 was elected leader, sending a full tablet report...
W20260806 06:41:32.140210 16713 tablet.cc:2369] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:32.183653 16888 raft_consensus.cc:482] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:32.183832 16888 raft_consensus.cc:504] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.184201 16888 leader_election.cc:288] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521), 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343)
I20260806 06:41:32.188141 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.184497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:35459 (local address 127.14.51.65:35343)
0806 06:41:32.184545 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.184547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.184558 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:32.184625 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.184626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.184633 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.184720 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.184898 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.185675 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187198 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.187766 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187849 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.187857 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.188034 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.188072 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:32.188141 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.184520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:49559 (local address 127.14.51.66:42521)
0806 06:41:32.184563 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.184565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.184692 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:41:32.184749 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.184750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.184759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.184815 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.184958 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.185680 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187198 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.187786 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187849 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.187857 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.188035 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.188072 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:32.188154 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.184418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:35343 (local address 127.14.51.67:35459)
0806 06:41:32.184479 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.184491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.184535 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.184743 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.184744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.184745 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.184819 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.184821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.185767 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.185767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.187087 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.187091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.187864 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.187961 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.187964 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.188015 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.188065 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":115}
I20260806 06:41:32.188165 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.184428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.14.51.67:49559)
0806 06:41:32.184610 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.184619 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.184673 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.184817 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.184818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.184830 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.184887 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.184888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.185767 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.185767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.187089 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.187091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.187864 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.187865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.187949 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.187953 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.188014 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.188065 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":58,"thread_start_us":116,"threads_started":1}
I20260806 06:41:32.188568 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be495c4fae3b44efa7d15af8ba69f317" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea" is_pre_election: true
I20260806 06:41:32.188771 16520 raft_consensus.cc:2454] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 0.
I20260806 06:41:32.188784 16666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be495c4fae3b44efa7d15af8ba69f317" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0364123266c450abbaa1fc323569a3c" is_pre_election: true
I20260806 06:41:32.188952 16666 raft_consensus.cc:2454] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 0.
I20260806 06:41:32.189020 16740 leader_election.cc:302] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, e5979826131144869ce09929102c9d6c; no voters:
I20260806 06:41:32.189159 16888 raft_consensus.cc:2781] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:32.189208 16888 raft_consensus.cc:482] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:32.189239 16888 raft_consensus.cc:3037] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.189929 16888 raft_consensus.cc:504] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.190073 16888 leader_election.cc:288] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 election: Requested vote from peers f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521), 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343)
I20260806 06:41:32.190208 16666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be495c4fae3b44efa7d15af8ba69f317" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0364123266c450abbaa1fc323569a3c"
I20260806 06:41:32.190301 16666 raft_consensus.cc:3037] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.190907 16666 raft_consensus.cc:2454] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 1.
I20260806 06:41:32.191032 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be495c4fae3b44efa7d15af8ba69f317" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea"
I20260806 06:41:32.191085 16739 leader_election.cc:302] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5979826131144869ce09929102c9d6c, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:32.191107 16520 raft_consensus.cc:3037] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.191203 16888 raft_consensus.cc:2781] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:32.191346 16888 raft_consensus.cc:686] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 1 LEADER]: Becoming Leader. State: Replica: e5979826131144869ce09929102c9d6c, State: Running, Role: LEADER
I20260806 06:41:32.191445 16888 consensus_queue.cc:237] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } }
I20260806 06:41:32.192045 16520 raft_consensus.cc:2454] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 1.
I20260806 06:41:32.192498 16358 catalog_manager.cc:5495] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5979826131144869ce09929102c9d6c (127.14.51.67). New cstate: current_term: 1 leader_uuid: "e5979826131144869ce09929102c9d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } health_report { overall_health: UNKNOWN } } }
W20260806 06:41:32.239113 16569 tablet.cc:2369] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:41:32 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998491, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:32.246651 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.239729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40339 (local address 127.0.0.1:45768)
0806 06:41:32.239919 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.239929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.240019 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.240171 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.240172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.240461 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:32.240511 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.240513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.241144 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.241145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.241655 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.241657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.242061 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.242062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.242121 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.242653 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:32.242657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:32.244080 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:32.245507 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.245515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.245517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.245706 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.245914 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:32.245915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:32.245916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:32.245970 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:32.246236 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:32.246243 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:32.246290 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.246377 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.246532 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":5,"thread_start_us":105,"threads_started":1}
I20260806 06:41:32.246662 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.239807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.14.51.126:40339)
0806 06:41:32.239885 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.239887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.240001 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:41:32.240087 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.240087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.240097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.240166 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:32.240592 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.241057 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.241748 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.241943 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.242061 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.244173 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:32.244175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:32.244175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:32.244181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:32.245404 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.245794 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.245795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.245795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.245848 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:32.246044 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:32.246045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:32.246045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:32.246148 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:32.246246 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.246400 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.246585 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":51}
I20260806 06:41:32.248519 16358 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:45768:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:41:32.248740 16358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:32.250350 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 9cabce8722c046d1b6370de8d7d652b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=300db17b1a3148e38dd9446f7c77357d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:32.250415 16646 tablet_service.cc:1467] Processing CreateTablet for tablet 9cabce8722c046d1b6370de8d7d652b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=300db17b1a3148e38dd9446f7c77357d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:32.250401 16791 tablet_service.cc:1467] Processing CreateTablet for tablet 9cabce8722c046d1b6370de8d7d652b0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=300db17b1a3148e38dd9446f7c77357d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:32.250521 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cabce8722c046d1b6370de8d7d652b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.250564 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cabce8722c046d1b6370de8d7d652b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.250576 16791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cabce8722c046d1b6370de8d7d652b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.251878 16881 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: Bootstrap starting.
I20260806 06:41:32.252038 16880 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: Bootstrap starting.
I20260806 06:41:32.252126 16882 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: Bootstrap starting.
I20260806 06:41:32.252359 16881 tablet_bootstrap.cc:654] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.252436 16880 tablet_bootstrap.cc:654] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.252652 16882 tablet_bootstrap.cc:654] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.253145 16880 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: No bootstrap required, opened a new log
I20260806 06:41:32.253199 16880 ts_tablet_manager.cc:1397] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:32.253334 16882 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: No bootstrap required, opened a new log
I20260806 06:41:32.253376 16882 ts_tablet_manager.cc:1397] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:32.253350 16880 raft_consensus.cc:348] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.253448 16880 raft_consensus.cc:374] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.253476 16880 raft_consensus.cc:729] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceafbce626f48cba7daa5b0c9089fea, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.253525 16880 consensus_queue.cc:260] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.253522 16882 raft_consensus.cc:348] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.253602 16882 raft_consensus.cc:374] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.253629 16882 raft_consensus.cc:729] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.253631 16880 ts_tablet_manager.cc:1428] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.253676 16882 consensus_queue.cc:260] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.253654 16886 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.253746 16886 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.253772 16882 ts_tablet_manager.cc:1428] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:32.253904 16887 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.253988 16887 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.254123 16881 tablet_bootstrap.cc:492] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: No bootstrap required, opened a new log
I20260806 06:41:32.254190 16881 ts_tablet_manager.cc:1397] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:32.254333 16881 raft_consensus.cc:348] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.254412 16881 raft_consensus.cc:374] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.254434 16881 raft_consensus.cc:729] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5979826131144869ce09929102c9d6c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.254473 16881 consensus_queue.cc:260] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.254542 16881 ts_tablet_manager.cc:1428] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.254601 16888 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.254710 16888 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 06:41:32.292074 16861 tablet.cc:2369] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:32.494726 16887 raft_consensus.cc:482] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:32.494819 16887 raft_consensus.cc:504] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.495191 16887 leader_election.cc:288] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:32.499106 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.495393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:57989 (local address 127.14.51.65:35343)
0806 06:41:32.495475 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.495478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.495575 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:41:32.495683 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.495684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.495694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.495759 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.495950 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.496708 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498220 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.498758 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498834 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.498842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.498997 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.499037 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:41:32.499106 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.495507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:47139 (local address 127.14.51.67:41973)
0806 06:41:32.495561 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.495562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.495573 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:32.495684 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.495685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.495694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.495760 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:32.495950 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.496701 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498220 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.498758 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498834 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.498842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.499008 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.499045 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:41:32.499159 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.495370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:41973 (local address 127.14.51.66:47139)
0806 06:41:32.495533 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.495542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.495590 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.495782 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.495783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.495784 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.495869 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.495870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.496784 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.496785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.498125 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.498128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.498844 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.498913 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.498918 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.498973 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.499023 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":100,"thread_start_us":79,"threads_started":1}
I20260806 06:41:32.499162 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.495335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:35343 (local address 127.14.51.66:57989)
0806 06:41:32.495482 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.495495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.495600 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.495781 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.495782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.495784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:32.495868 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.495870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.496786 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.496786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.498125 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.498128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.498845 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.498845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.498915 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.498917 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.498973 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.499023 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":93,"thread_start_us":77,"threads_started":1}
I20260806 06:41:32.499534 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cabce8722c046d1b6370de8d7d652b0" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea" is_pre_election: true
I20260806 06:41:32.499635 16520 raft_consensus.cc:2454] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:32.499707 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cabce8722c046d1b6370de8d7d652b0" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c" is_pre_election: true
I20260806 06:41:32.499822 16595 leader_election.cc:302] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:32.499857 16812 raft_consensus.cc:2454] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:32.499951 16887 raft_consensus.cc:2781] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:32.499989 16887 raft_consensus.cc:482] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:32.500008 16887 raft_consensus.cc:3037] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.500726 16887 raft_consensus.cc:504] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.500864 16887 leader_election.cc:288] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Requested vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:32.501008 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cabce8722c046d1b6370de8d7d652b0" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea"
I20260806 06:41:32.501025 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cabce8722c046d1b6370de8d7d652b0" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c"
I20260806 06:41:32.501086 16520 raft_consensus.cc:3037] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.501099 16812 raft_consensus.cc:3037] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.501904 16520 raft_consensus.cc:2454] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:32.501904 16812 raft_consensus.cc:2454] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:32.502075 16595 leader_election.cc:302] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:32.502188 16887 raft_consensus.cc:2781] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:32.502327 16887 raft_consensus.cc:686] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [term 1 LEADER]: Becoming Leader. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Running, Role: LEADER
I20260806 06:41:32.502424 16887 consensus_queue.cc:237] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.502897 16903 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: TxnStatusTablet state changed. Reason: New leader f0364123266c450abbaa1fc323569a3c. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.503000 16903 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:32.503249 16905 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:41:32.503360 16905 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:41:32.503290 16356 catalog_manager.cc:5495] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c reported cstate change: term changed from 0 to 1, leader changed from <none> to f0364123266c450abbaa1fc323569a3c (127.14.51.66). New cstate: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:32.538507 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.536144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.14.51.66:42521)
0806 06:41:32.536198 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.536200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.536212 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:41:32.536280 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.536281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.536303 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.536374 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.536507 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.536987 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.537714 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.537913 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.537995 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.538182 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.538282 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.538371 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.538412 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.538432 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:41:32.538519 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.536063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.0.0.1:48386)
0806 06:41:32.536142 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.536155 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.536216 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.536383 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.536383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.536385 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:32.536432 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.536434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.537090 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.537091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.537630 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.537632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.537997 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.537998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.538061 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.538075 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.538349 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.538358 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.538406 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.538453 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:41:32.540400 16812 raft_consensus.cc:1261] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.540400 16520 raft_consensus.cc:1261] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.540730 16903 consensus_queue.cc:1035] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:32.540838 16887 consensus_queue.cc:1035] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:32.541909 16888 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: TxnStatusTablet state changed. Reason: New leader f0364123266c450abbaa1fc323569a3c. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.541998 16888 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.541978 16886 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: TxnStatusTablet state changed. Reason: New leader f0364123266c450abbaa1fc323569a3c. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.542058 16886 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.542949 16887 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.543062 16887 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:32.543584 16888 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.543669 16888 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P e5979826131144869ce09929102c9d6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.543833 16911 mvcc.cc:205] Tried to move back new op lower bound from 7315449825442971648 to 7315449825290633216. Current Snapshot: MvccSnapshot[applied={T|T < 7315449825442971648}]
I20260806 06:41:32.543895 16887 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.543954 16908 mvcc.cc:205] Tried to move back new op lower bound from 7315449825442971648 to 7315449825290633216. Current Snapshot: MvccSnapshot[applied={T|T < 7315449825442971648}]
I20260806 06:41:32.543996 16887 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P f0364123266c450abbaa1fc323569a3c: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:32.544622 16886 tablet_replica.cc:439] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } } }
I20260806 06:41:32.544713 16886 tablet_replica.cc:442] T 9cabce8722c046d1b6370de8d7d652b0 P 9ceafbce626f48cba7daa5b0c9089fea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:32.545406 16909 mvcc.cc:205] Tried to move back new op lower bound from 7315449825442971648 to 7315449825290633216. Current Snapshot: MvccSnapshot[applied={T|T < 7315449825442971648}]
I20260806 06:41:32.550098 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.547754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.14.51.67:41973)
0806 06:41:32.547795 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.547797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.547806 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:32.547889 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.547890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.547907 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.547973 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.548131 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.548564 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.549332 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.549532 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.549604 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.549779 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.549870 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.549960 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.549998 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.550019 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:32.555054 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.552513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:57101 (local address 127.14.51.66:42521)
0806 06:41:32.552550 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.552552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.552730 (+ 178us) server_negotiation.cc:407] Connection header received
0806 06:41:32.552771 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.552772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.552779 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.552823 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.552923 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.553408 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.554469 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.554632 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.554674 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.554877 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.554898 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.554930 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.554993 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.555005 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:41:32.555135 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.552473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.14.51.67:57101)
0806 06:41:32.552648 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.552659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.552721 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.552828 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.552829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.552830 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:32.552871 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.552872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.553493 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.553494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.554384 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.554387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.554702 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.554704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.554770 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.554784 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.554945 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.554946 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.554993 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.555037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260806 06:41:32.557430 16666 raft_consensus.cc:1261] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Refusing update from remote peer e5979826131144869ce09929102c9d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.557622 16520 raft_consensus.cc:1261] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Refusing update from remote peer e5979826131144869ce09929102c9d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.557749 16890 consensus_queue.cc:1035] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:32.557859 16888 consensus_queue.cc:1035] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:32.566444 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.563696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:48647 (local address 127.14.51.67:41973)
0806 06:41:32.563730 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.563732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.564060 (+ 328us) server_negotiation.cc:407] Connection header received
0806 06:41:32.564098 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.564099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.564107 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.564146 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.564266 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.564687 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.565810 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.566029 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.566099 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.566282 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.566305 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.566342 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.566385 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.566400 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:32.566507 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.563641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:41973 (local address 127.14.51.66:48647)
0806 06:41:32.563810 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.563821 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.564050 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.564160 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.564160 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.564162 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:32.564214 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.564215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.564759 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.564760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.565691 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.565694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.566119 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.566120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.566188 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.566201 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.566344 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.566345 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.566386 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.566425 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260806 06:41:32.575640 16358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:32.575750 16358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:32.576462 16358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Sending DeleteTablet for 3 replicas of tablet be495c4fae3b44efa7d15af8ba69f317
I20260806 06:41:32.576778 16791 tablet_service.cc:1514] Processing DeleteTablet for tablet be495c4fae3b44efa7d15af8ba69f317 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:32 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:38134
I20260806 06:41:32.576901 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet be495c4fae3b44efa7d15af8ba69f317 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:32 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34098
I20260806 06:41:32.577191 16931 tablet_replica.cc:330] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: stopping tablet replica
I20260806 06:41:32.577332 16931 raft_consensus.cc:2229] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:32.577486 16931 raft_consensus.cc:2258] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:32.577527 16646 tablet_service.cc:1514] Processing DeleteTablet for tablet be495c4fae3b44efa7d15af8ba69f317 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:32 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48382
I20260806 06:41:32.577723 16358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:32.577872 16358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:32.578171 16931 ts_tablet_manager.cc:1905] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:32.579053 16791 tablet_service.cc:1467] Processing CreateTablet for tablet 7f45494866364d9ca475d205f13088da (DEFAULT_TABLE table=test-table [id=b87f924a0eee477b94721d9a48f78899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.579219 16791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f45494866364d9ca475d205f13088da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.580498 16931 ts_tablet_manager.cc:1918] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:32.580514 16881 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Bootstrap starting.
I20260806 06:41:32.580806 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 7f45494866364d9ca475d205f13088da (DEFAULT_TABLE table=test-table [id=b87f924a0eee477b94721d9a48f78899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.580914 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f45494866364d9ca475d205f13088da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.581033 16881 tablet_bootstrap.cc:654] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.581440 16931 log.cc:1196] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/be495c4fae3b44efa7d15af8ba69f317
I20260806 06:41:32.581827 16931 ts_tablet_manager.cc:1939] T be495c4fae3b44efa7d15af8ba69f317 P e5979826131144869ce09929102c9d6c: Deleting consensus metadata
I20260806 06:41:32.582253 16341 catalog_manager.cc:4841] TS e5979826131144869ce09929102c9d6c (127.14.51.67:41973): tablet be495c4fae3b44efa7d15af8ba69f317 (table test-table [id=550833730abd4bccb3fd233fde670b8d]) successfully deleted
I20260806 06:41:32.582715 16881 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: No bootstrap required, opened a new log
I20260806 06:41:32.582760 16880 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Bootstrap starting.
I20260806 06:41:32.582762 16881 ts_tablet_manager.cc:1397] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:32.582911 16881 raft_consensus.cc:348] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.582975 16881 raft_consensus.cc:374] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.582994 16881 raft_consensus.cc:729] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5979826131144869ce09929102c9d6c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.583032 16881 consensus_queue.cc:260] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.583149 16881 ts_tablet_manager.cc:1428] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.583158 16880 tablet_bootstrap.cc:654] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.583813 16646 tablet_service.cc:1467] Processing CreateTablet for tablet 7f45494866364d9ca475d205f13088da (DEFAULT_TABLE table=test-table [id=b87f924a0eee477b94721d9a48f78899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:32.583985 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f45494866364d9ca475d205f13088da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:32.584952 16882 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Bootstrap starting.
I20260806 06:41:32.585378 16882 tablet_bootstrap.cc:654] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:32.585728 16932 tablet_replica.cc:330] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: stopping tablet replica
I20260806 06:41:32.585832 16932 raft_consensus.cc:2229] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:32.585889 16932 raft_consensus.cc:2258] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:32.586050 16933 tablet_replica.cc:330] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: stopping tablet replica
I20260806 06:41:32.586153 16933 raft_consensus.cc:2229] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:32.586208 16933 raft_consensus.cc:2258] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:32.586365 16882 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: No bootstrap required, opened a new log
I20260806 06:41:32.586402 16882 ts_tablet_manager.cc:1397] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:32.586524 16882 raft_consensus.cc:348] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.586591 16933 ts_tablet_manager.cc:1905] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:32.586624 16882 raft_consensus.cc:374] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.586654 16882 raft_consensus.cc:729] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.586699 16882 consensus_queue.cc:260] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.586787 16882 ts_tablet_manager.cc:1428] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:32.587105 16932 ts_tablet_manager.cc:1905] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:32.587499 16880 tablet_bootstrap.cc:492] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: No bootstrap required, opened a new log
I20260806 06:41:32.587558 16880 ts_tablet_manager.cc:1397] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 06:41:32.587688 16880 raft_consensus.cc:348] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.587774 16880 raft_consensus.cc:374] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:32.587797 16880 raft_consensus.cc:729] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceafbce626f48cba7daa5b0c9089fea, State: Initialized, Role: FOLLOWER
I20260806 06:41:32.587844 16880 consensus_queue.cc:260] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.587939 16880 ts_tablet_manager.cc:1428] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:32.588980 16932 ts_tablet_manager.cc:1918] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:32.589080 16932 log.cc:1196] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/be495c4fae3b44efa7d15af8ba69f317
I20260806 06:41:32.589378 16932 ts_tablet_manager.cc:1939] T be495c4fae3b44efa7d15af8ba69f317 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting consensus metadata
I20260806 06:41:32.589851 16343 catalog_manager.cc:4841] TS 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343): tablet be495c4fae3b44efa7d15af8ba69f317 (table test-table [id=550833730abd4bccb3fd233fde670b8d]) successfully deleted
I20260806 06:41:32.592126 16933 ts_tablet_manager.cc:1918] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:32.592207 16933 log.cc:1196] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/be495c4fae3b44efa7d15af8ba69f317
I20260806 06:41:32.592517 16933 ts_tablet_manager.cc:1939] T be495c4fae3b44efa7d15af8ba69f317 P f0364123266c450abbaa1fc323569a3c: Deleting consensus metadata
I20260806 06:41:32.592895 16342 catalog_manager.cc:4841] TS f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521): tablet be495c4fae3b44efa7d15af8ba69f317 (table test-table [id=550833730abd4bccb3fd233fde670b8d]) successfully deleted
I20260806 06:41:32.642192 16922 raft_consensus.cc:482] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:32.642295 16922 raft_consensus.cc:504] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.642477 16922 leader_election.cc:288] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:32.642714 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f45494866364d9ca475d205f13088da" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c" is_pre_election: true
I20260806 06:41:32.642730 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f45494866364d9ca475d205f13088da" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea" is_pre_election: true
I20260806 06:41:32.642805 16812 raft_consensus.cc:2454] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:32.642844 16520 raft_consensus.cc:2454] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:32.642952 16593 leader_election.cc:302] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5979826131144869ce09929102c9d6c, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:32.643054 16922 raft_consensus.cc:2781] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:32.643080 16922 raft_consensus.cc:482] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:32.643105 16922 raft_consensus.cc:3037] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.644028 16922 raft_consensus.cc:504] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.644158 16922 leader_election.cc:288] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Requested vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:32.644335 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f45494866364d9ca475d205f13088da" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea"
I20260806 06:41:32.644403 16520 raft_consensus.cc:3037] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.644405 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f45494866364d9ca475d205f13088da" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c"
I20260806 06:41:32.644488 16812 raft_consensus.cc:3037] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:32.645161 16520 raft_consensus.cc:2454] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:32.645305 16812 raft_consensus.cc:2454] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:32.645323 16595 leader_election.cc:302] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:32.645397 16922 raft_consensus.cc:2781] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:32.645432 16922 raft_consensus.cc:686] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 1 LEADER]: Becoming Leader. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Running, Role: LEADER
I20260806 06:41:32.645473 16922 consensus_queue.cc:237] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:32.646016 16358 catalog_manager.cc:5495] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c reported cstate change: term changed from 0 to 1, leader changed from <none> to f0364123266c450abbaa1fc323569a3c (127.14.51.66). New cstate: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:32.664834 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.662469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.14.51.66:42521)
0806 06:41:32.662535 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.662537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.662547 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:32.662570 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.662571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.662581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.662641 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.662799 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.663333 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.664131 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.664379 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.664453 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.664596 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.664629 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.664677 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.664735 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.664752 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:41:32.669065 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.666999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:45557 (local address 127.14.51.66:42521)
0806 06:41:32.667033 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:32.667035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:32.667043 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:32.667107 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:32.667108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:32.667116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:32.667197 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:32.667293 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.667713 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.668412 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.668593 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.668666 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.668807 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.668838 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.668898 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:32.668958 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:41:32.668973 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:41:32.669219 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:32.666931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.14.51.66:45557)
0806 06:41:32.666999 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:32.667009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:32.667051 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:32.667199 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:32.667200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:32.667201 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:32.667234 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.667235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.667794 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.667795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.668313 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:32.668314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:32.668666 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:32.668666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:32.668717 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:32.668727 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:32.668900 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:32.668902 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:32.668954 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:41:32.668998 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:41:32.671168 16812 raft_consensus.cc:1261] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.671315 16520 raft_consensus.cc:1261] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:32.671422 16922 consensus_queue.cc:1035] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:32.671519 16887 consensus_queue.cc:1035] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:41:32.676656 16908 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:41:34.194659 16357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:34.194775 16357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:34.195660 16357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Sending DeleteTablet for 3 replicas of tablet 7f45494866364d9ca475d205f13088da
I20260806 06:41:34.195938 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f45494866364d9ca475d205f13088da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34098
I20260806 06:41:34.195994 16646 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f45494866364d9ca475d205f13088da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48382
I20260806 06:41:34.196144 16962 tablet_replica.cc:330] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: stopping tablet replica
I20260806 06:41:34.196193 16962 raft_consensus.cc:2229] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:34.196233 16962 raft_consensus.cc:2258] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.196352 16963 tablet_replica.cc:330] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: stopping tablet replica
I20260806 06:41:34.196403 16963 raft_consensus.cc:2229] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:34.196480 16963 raft_consensus.cc:2258] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.196504 16962 ts_tablet_manager.cc:1905] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:34.196688 16963 ts_tablet_manager.cc:1905] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:34.198297 16963 ts_tablet_manager.cc:1918] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:41:34.198352 16963 log.cc:1196] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/7f45494866364d9ca475d205f13088da
I20260806 06:41:34.198539 16962 ts_tablet_manager.cc:1918] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:41:34.198599 16962 log.cc:1196] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/7f45494866364d9ca475d205f13088da
I20260806 06:41:34.198849 16962 ts_tablet_manager.cc:1939] T 7f45494866364d9ca475d205f13088da P 9ceafbce626f48cba7daa5b0c9089fea: Deleting consensus metadata
I20260806 06:41:34.198853 16963 ts_tablet_manager.cc:1939] T 7f45494866364d9ca475d205f13088da P f0364123266c450abbaa1fc323569a3c: Deleting consensus metadata
I20260806 06:41:34.199164 16791 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f45494866364d9ca475d205f13088da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:38134
I20260806 06:41:34.199245 16342 catalog_manager.cc:4841] TS f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521): tablet 7f45494866364d9ca475d205f13088da (table test-table [id=b87f924a0eee477b94721d9a48f78899]) successfully deleted
I20260806 06:41:34.199256 16343 catalog_manager.cc:4841] TS 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343): tablet 7f45494866364d9ca475d205f13088da (table test-table [id=b87f924a0eee477b94721d9a48f78899]) successfully deleted
I20260806 06:41:34.199349 16965 tablet_replica.cc:330] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: stopping tablet replica
I20260806 06:41:34.199400 16965 raft_consensus.cc:2229] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:34.199450 16965 raft_consensus.cc:2258] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.199663 16965 ts_tablet_manager.cc:1905] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:34.200903 16965 ts_tablet_manager.cc:1918] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:41:34.200982 16965 log.cc:1196] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/7f45494866364d9ca475d205f13088da
I20260806 06:41:34.201169 16965 ts_tablet_manager.cc:1939] T 7f45494866364d9ca475d205f13088da P e5979826131144869ce09929102c9d6c: Deleting consensus metadata
I20260806 06:41:34.201642 16341 catalog_manager.cc:4841] TS e5979826131144869ce09929102c9d6c (127.14.51.67:41973): tablet 7f45494866364d9ca475d205f13088da (table test-table [id=b87f924a0eee477b94721d9a48f78899]) successfully deleted
Aug 06 06:41:34 dist-test-slave-1xrh krb5kdc[16309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998494, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:41:34 dist-test-slave-1xrh krb5kdc[16309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998494, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:34.225142 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:34.212933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.14.51.126:40339)
0806 06:41:34.213101 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:34.213103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:34.213112 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:34.213201 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:34.213202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:34.213219 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:34.213292 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:34.213838 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.214334 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.215048 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.215305 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.215383 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:34.222535 (+ 7152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:34.222537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:34.222537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:34.222544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:34.223992 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:34.224347 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:34.224348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:34.224349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:34.224409 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:34.224612 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:34.224613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:34.224613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:34.224727 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:34.224837 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:34.224939 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:41:34.225028 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260806 06:41:34.226430 16357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:34.226573 16357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:34.228008 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 (DEFAULT_TABLE table=test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.228032 16646 tablet_service.cc:1467] Processing CreateTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 (DEFAULT_TABLE table=test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.228070 16791 tablet_service.cc:1467] Processing CreateTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 (DEFAULT_TABLE table=test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.228171 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bcb2d53e4f14e6cad9af44f34f69833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.228173 16791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bcb2d53e4f14e6cad9af44f34f69833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.228175 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bcb2d53e4f14e6cad9af44f34f69833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.229095 16973 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Bootstrap starting.
I20260806 06:41:34.229383 16973 tablet_bootstrap.cc:654] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.229629 16974 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Bootstrap starting.
I20260806 06:41:34.229709 16975 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Bootstrap starting.
I20260806 06:41:34.229926 16974 tablet_bootstrap.cc:654] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.230109 16975 tablet_bootstrap.cc:654] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.230315 16973 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: No bootstrap required, opened a new log
I20260806 06:41:34.230413 16973 ts_tablet_manager.cc:1397] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:34.230500 16974 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: No bootstrap required, opened a new log
I20260806 06:41:34.230542 16974 ts_tablet_manager.cc:1397] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:34.230556 16973 raft_consensus.cc:348] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.230628 16973 raft_consensus.cc:374] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.230654 16973 raft_consensus.cc:729] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceafbce626f48cba7daa5b0c9089fea, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.230651 16974 raft_consensus.cc:348] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.230719 16974 raft_consensus.cc:374] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.230731 16975 tablet_bootstrap.cc:492] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: No bootstrap required, opened a new log
I20260806 06:41:34.230744 16974 raft_consensus.cc:729] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5979826131144869ce09929102c9d6c, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.230712 16973 consensus_queue.cc:260] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.230777 16975 ts_tablet_manager.cc:1397] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:34.230799 16974 consensus_queue.cc:260] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.230881 16973 ts_tablet_manager.cc:1428] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:34.230893 16975 raft_consensus.cc:348] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.230964 16974 ts_tablet_manager.cc:1428] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:34.230990 16975 raft_consensus.cc:374] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.231153 16975 raft_consensus.cc:729] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.231215 16975 consensus_queue.cc:260] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.231316 16975 ts_tablet_manager.cc:1428] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:34.344242 16980 raft_consensus.cc:482] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:34.344331 16980 raft_consensus.cc:504] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.344486 16980 leader_election.cc:288] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521)
I20260806 06:41:34.344677 16666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bcb2d53e4f14e6cad9af44f34f69833" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0364123266c450abbaa1fc323569a3c" is_pre_election: true
I20260806 06:41:34.344667 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bcb2d53e4f14e6cad9af44f34f69833" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea" is_pre_election: true
I20260806 06:41:34.344770 16666 raft_consensus.cc:2454] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 0.
I20260806 06:41:34.344770 16520 raft_consensus.cc:2454] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 0.
I20260806 06:41:34.344935 16740 leader_election.cc:302] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, e5979826131144869ce09929102c9d6c; no voters:
I20260806 06:41:34.345103 16980 raft_consensus.cc:2781] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:34.345146 16980 raft_consensus.cc:482] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:34.345160 16980 raft_consensus.cc:3037] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.345916 16980 raft_consensus.cc:504] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.346069 16980 leader_election.cc:288] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 election: Requested vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521)
I20260806 06:41:34.346201 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bcb2d53e4f14e6cad9af44f34f69833" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea"
I20260806 06:41:34.346222 16666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bcb2d53e4f14e6cad9af44f34f69833" candidate_uuid: "e5979826131144869ce09929102c9d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0364123266c450abbaa1fc323569a3c"
I20260806 06:41:34.346277 16520 raft_consensus.cc:3037] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.346288 16666 raft_consensus.cc:3037] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.347039 16520 raft_consensus.cc:2454] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 1.
I20260806 06:41:34.347039 16666 raft_consensus.cc:2454] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5979826131144869ce09929102c9d6c in term 1.
I20260806 06:41:34.347225 16740 leader_election.cc:302] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, e5979826131144869ce09929102c9d6c; no voters:
I20260806 06:41:34.347343 16980 raft_consensus.cc:2781] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:34.347536 16980 raft_consensus.cc:686] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 1 LEADER]: Becoming Leader. State: Replica: e5979826131144869ce09929102c9d6c, State: Running, Role: LEADER
I20260806 06:41:34.347585 16980 consensus_queue.cc:237] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.348299 16358 catalog_manager.cc:5495] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5979826131144869ce09929102c9d6c (127.14.51.67). New cstate: current_term: 1 leader_uuid: "e5979826131144869ce09929102c9d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } health_report { overall_health: HEALTHY } } }
I20260806 06:41:34.378695 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:34.376028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.14.51.67:41973)
0806 06:41:34.376187 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:34.376188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:34.376194 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:41:34.376212 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:34.376213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:34.376223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:34.376354 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:34.376495 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.376986 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.377742 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.378094 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.378205 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:34.378391 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:34.378436 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:34.378501 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:34.378565 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:41:34.378588 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260806 06:41:34.381070 16666 raft_consensus.cc:1261] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Refusing update from remote peer e5979826131144869ce09929102c9d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:34.381100 16520 raft_consensus.cc:1261] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Refusing update from remote peer e5979826131144869ce09929102c9d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:34.381302 16980 consensus_queue.cc:1035] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:34.381408 16981 consensus_queue.cc:1035] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:34.391578 16357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:34.391674 16357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:34.392295 16357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Sending DeleteTablet for 3 replicas of tablet 8bcb2d53e4f14e6cad9af44f34f69833
I20260806 06:41:34.392514 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34098
I20260806 06:41:34.392580 16791 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:38134
I20260806 06:41:34.392613 16962 tablet_replica.cc:330] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: stopping tablet replica
I20260806 06:41:34.392665 16965 tablet_replica.cc:330] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: stopping tablet replica
I20260806 06:41:34.392669 16962 raft_consensus.cc:2229] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:34.392719 16962 raft_consensus.cc:2258] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.392716 16965 raft_consensus.cc:2229] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:34.392728 16646 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bcb2d53e4f14e6cad9af44f34f69833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:34 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48382
I20260806 06:41:34.392774 16965 raft_consensus.cc:2258] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.392817 16963 tablet_replica.cc:330] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: stopping tablet replica
I20260806 06:41:34.392875 16963 raft_consensus.cc:2229] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:34.392913 16963 raft_consensus.cc:2258] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:34.393055 16962 ts_tablet_manager.cc:1905] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:34.393095 16965 ts_tablet_manager.cc:1905] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:34.393095 16357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:41:34.393219 16963 ts_tablet_manager.cc:1905] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:41:34.393229 16357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:34.394394 16500 tablet_service.cc:1467] Processing CreateTablet for tablet ceb4e250a5454e83902a8506d36395a8 (DEFAULT_TABLE table=test-table [id=51782ab2ac9f47c49524c710e7e2dd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.394493 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb4e250a5454e83902a8506d36395a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.394873 16963 ts_tablet_manager.cc:1918] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:34.394865 16965 ts_tablet_manager.cc:1918] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:34.394870 16791 tablet_service.cc:1467] Processing CreateTablet for tablet ceb4e250a5454e83902a8506d36395a8 (DEFAULT_TABLE table=test-table [id=51782ab2ac9f47c49524c710e7e2dd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.394929 16963 log.cc:1196] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/8bcb2d53e4f14e6cad9af44f34f69833
I20260806 06:41:34.394932 16965 log.cc:1196] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/8bcb2d53e4f14e6cad9af44f34f69833
I20260806 06:41:34.394972 16646 tablet_service.cc:1467] Processing CreateTablet for tablet ceb4e250a5454e83902a8506d36395a8 (DEFAULT_TABLE table=test-table [id=51782ab2ac9f47c49524c710e7e2dd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:34.395022 16791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb4e250a5454e83902a8506d36395a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.395067 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb4e250a5454e83902a8506d36395a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:34.395193 16965 ts_tablet_manager.cc:1939] T 8bcb2d53e4f14e6cad9af44f34f69833 P e5979826131144869ce09929102c9d6c: Deleting consensus metadata
I20260806 06:41:34.395203 16963 ts_tablet_manager.cc:1939] T 8bcb2d53e4f14e6cad9af44f34f69833 P f0364123266c450abbaa1fc323569a3c: Deleting consensus metadata
I20260806 06:41:34.396153 16974 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Bootstrap starting.
I20260806 06:41:34.396194 16975 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: Bootstrap starting.
I20260806 06:41:34.396245 16341 catalog_manager.cc:4841] TS e5979826131144869ce09929102c9d6c (127.14.51.67:41973): tablet 8bcb2d53e4f14e6cad9af44f34f69833 (table test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]) successfully deleted
I20260806 06:41:34.396402 16342 catalog_manager.cc:4841] TS f0364123266c450abbaa1fc323569a3c (127.14.51.66:42521): tablet 8bcb2d53e4f14e6cad9af44f34f69833 (table test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]) successfully deleted
I20260806 06:41:34.396423 16962 ts_tablet_manager.cc:1918] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:34.396466 16974 tablet_bootstrap.cc:654] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.396476 16962 log.cc:1196] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/8bcb2d53e4f14e6cad9af44f34f69833
I20260806 06:41:34.396540 16975 tablet_bootstrap.cc:654] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.396720 16962 ts_tablet_manager.cc:1939] T 8bcb2d53e4f14e6cad9af44f34f69833 P 9ceafbce626f48cba7daa5b0c9089fea: Deleting consensus metadata
I20260806 06:41:34.397020 16974 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: No bootstrap required, opened a new log
I20260806 06:41:34.397027 16343 catalog_manager.cc:4841] TS 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343): tablet 8bcb2d53e4f14e6cad9af44f34f69833 (table test-table [id=0cdd2152a2cc499eb3f84a8977d3fd94]) successfully deleted
I20260806 06:41:34.397084 16974 ts_tablet_manager.cc:1397] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:34.397099 16975 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: No bootstrap required, opened a new log
I20260806 06:41:34.397137 16975 ts_tablet_manager.cc:1397] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:34.397187 16974 raft_consensus.cc:348] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.397249 16974 raft_consensus.cc:374] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.397269 16974 raft_consensus.cc:729] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5979826131144869ce09929102c9d6c, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.397233 16975 raft_consensus.cc:348] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.397293 16975 raft_consensus.cc:374] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.397344 16975 raft_consensus.cc:729] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.397308 16974 consensus_queue.cc:260] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.397389 16975 consensus_queue.cc:260] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.397421 16974 ts_tablet_manager.cc:1428] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:34.397480 16975 ts_tablet_manager.cc:1428] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:34.398036 16973 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: Bootstrap starting.
I20260806 06:41:34.398743 16973 tablet_bootstrap.cc:654] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:34.400012 16973 tablet_bootstrap.cc:492] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: No bootstrap required, opened a new log
I20260806 06:41:34.400077 16973 ts_tablet_manager.cc:1397] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:41:34.400252 16973 raft_consensus.cc:348] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.400331 16973 raft_consensus.cc:374] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:34.400355 16973 raft_consensus.cc:729] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceafbce626f48cba7daa5b0c9089fea, State: Initialized, Role: FOLLOWER
I20260806 06:41:34.400403 16973 consensus_queue.cc:260] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.400489 16973 ts_tablet_manager.cc:1428] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:34.422657 16922 raft_consensus.cc:482] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:34.422755 16922 raft_consensus.cc:504] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.422910 16922 leader_election.cc:288] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:34.423038 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb4e250a5454e83902a8506d36395a8" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea" is_pre_election: true
I20260806 06:41:34.423105 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb4e250a5454e83902a8506d36395a8" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c" is_pre_election: true
I20260806 06:41:34.423177 16520 raft_consensus.cc:2454] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:34.423209 16812 raft_consensus.cc:2454] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 0.
I20260806 06:41:34.423359 16595 leader_election.cc:302] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ceafbce626f48cba7daa5b0c9089fea, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:34.423488 16922 raft_consensus.cc:2781] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:34.423539 16922 raft_consensus.cc:482] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:34.423568 16922 raft_consensus.cc:3037] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.424165 16922 raft_consensus.cc:504] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.424306 16922 leader_election.cc:288] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Requested vote from peers 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343), e5979826131144869ce09929102c9d6c (127.14.51.67:41973)
I20260806 06:41:34.424463 16812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb4e250a5454e83902a8506d36395a8" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5979826131144869ce09929102c9d6c"
I20260806 06:41:34.424479 16520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb4e250a5454e83902a8506d36395a8" candidate_uuid: "f0364123266c450abbaa1fc323569a3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ceafbce626f48cba7daa5b0c9089fea"
I20260806 06:41:34.424535 16812 raft_consensus.cc:3037] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.424553 16520 raft_consensus.cc:3037] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:34.425199 16520 raft_consensus.cc:2454] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:34.425199 16812 raft_consensus.cc:2454] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0364123266c450abbaa1fc323569a3c in term 1.
I20260806 06:41:34.425374 16593 leader_election.cc:302] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5979826131144869ce09929102c9d6c, f0364123266c450abbaa1fc323569a3c; no voters:
I20260806 06:41:34.425514 16922 raft_consensus.cc:2781] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:34.425590 16922 raft_consensus.cc:686] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 1 LEADER]: Becoming Leader. State: Replica: f0364123266c450abbaa1fc323569a3c, State: Running, Role: LEADER
I20260806 06:41:34.425650 16922 consensus_queue.cc:237] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } }
I20260806 06:41:34.426146 16358 catalog_manager.cc:5495] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c reported cstate change: term changed from 0 to 1, leader changed from <none> to f0364123266c450abbaa1fc323569a3c (127.14.51.66). New cstate: current_term: 1 leader_uuid: "f0364123266c450abbaa1fc323569a3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0364123266c450abbaa1fc323569a3c" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:34.439157 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:34.436787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.14.51.66:42521)
0806 06:41:34.436947 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:34.436949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:34.436956 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:34.436977 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:34.436977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:34.436995 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:34.437071 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:34.437213 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.437711 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.438424 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:34.438643 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:34.438718 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:34.438857 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:34.438891 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:34.438972 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:34.439032 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:41:34.439053 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260806 06:41:34.441309 16520 raft_consensus.cc:1261] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:34.441372 16812 raft_consensus.cc:1261] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Refusing update from remote peer f0364123266c450abbaa1fc323569a3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:34.441560 16922 consensus_queue.cc:1035] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5979826131144869ce09929102c9d6c" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:34.441686 16944 consensus_queue.cc:1035] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ceafbce626f48cba7daa5b0c9089fea" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:41:34.446012 16942 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:41:35.965938 16357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:35.966048 16357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:35.966818 16357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fdebf5ab8794689bcb706621ae1aad4: Sending DeleteTablet for 3 replicas of tablet ceb4e250a5454e83902a8506d36395a8
I20260806 06:41:35.967160 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb4e250a5454e83902a8506d36395a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:35 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34098
I20260806 06:41:35.967212 16644 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb4e250a5454e83902a8506d36395a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:35 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48382
I20260806 06:41:35.967379 16791 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb4e250a5454e83902a8506d36395a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:35 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:38134
I20260806 06:41:35.967454 17018 tablet_replica.cc:330] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea: stopping tablet replica
I20260806 06:41:35.967432 17019 tablet_replica.cc:330] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: stopping tablet replica
I20260806 06:41:35.967542 17018 raft_consensus.cc:2229] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:35.967592 17018 raft_consensus.cc:2258] T ceb4e250a5454e83902a8506d36395a8 P 9ceafbce626f48cba7daa5b0c9089fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:35.967594 17020 tablet_replica.cc:330] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: stopping tablet replica
I20260806 06:41:35.967648 17020 raft_consensus.cc:2229] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:35.967696 17020 raft_consensus.cc:2258] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:35.967680 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 16431
I20260806 06:41:35.969220 17020 ts_tablet_manager.cc:1905] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:35.969818 17019 raft_consensus.cc:2229] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:35.969980 17019 raft_consensus.cc:2258] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:35.970360 17019 ts_tablet_manager.cc:1905] T ceb4e250a5454e83902a8506d36395a8 P f0364123266c450abbaa1fc323569a3c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:41:35.976966 16343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:35343 (error 108)
I20260806 06:41:35.977566 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 16577
I20260806 06:41:35.977662 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:35.977367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:35343 (local address 127.0.0.1:34112)
0806 06:41:35.977485 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:35.977601 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":78,"threads_started":1}
W20260806 06:41:35.978464 16343 catalog_manager.cc:4584] TS 9ceafbce626f48cba7daa5b0c9089fea (127.14.51.65:35343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ceb4e250a5454e83902a8506d36395a8: Network error: Client connection negotiation failed: client connection to 127.14.51.65:35343: connect: Connection refused (error 111)
I20260806 06:41:35.981770 17020 ts_tablet_manager.cc:1918] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:41:35.981843 17020 log.cc:1196] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/ceb4e250a5454e83902a8506d36395a8
I20260806 06:41:35.982075 17020 ts_tablet_manager.cc:1939] T ceb4e250a5454e83902a8506d36395a8 P e5979826131144869ce09929102c9d6c: Deleting consensus metadata
I20260806 06:41:35.985373 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 16721
I20260806 06:41:35.985394 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:35.985265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42521 (local address 127.0.0.1:48404)
0806 06:41:35.985325 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:35.985355 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.66:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:41:36.007840 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 16322
I20260806 06:41:36.007941 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:36.007786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:41973 (local address 127.0.0.1:38162)
0806 06:41:36.007860 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:36.007898 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.67:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
2026-08-06T06:41:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5740 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:37 dist-test-slave-1xrh krb5kdc[17026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:37 dist-test-slave-1xrh krb5kdc[17026](info): set up 2 sockets
Aug 06 06:41:37 dist-test-slave-1xrh krb5kdc[17026](info): commencing operation
krb5kdc: starting...
W20260806 06:41:38.154126 14541 mini_kdc.cc:121] Time spent starting KDC: real 2.124s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:38.269788 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43361
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:38.349709 17040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:38.349893 17040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:38.349913 17040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:38.350998 17040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:38.351035 17040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:38.351048 17040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:38.351059 17040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:38.351110 17040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:38.352520 17040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:38.352717 17040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:38.352916 17040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:38.357890 17045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.357903 17046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.357913 17048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.357932 17040 server_base.cc:1034] running on GCE node
I20260806 06:41:38.358325 17040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:38.358557 17040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:38.359690 17040 hybrid_clock.cc:648] HybridClock initialized: now 1785998498359670 us; error 38 us; skew 500 ppm
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:38.362223 17040 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:38.362694 17040 webserver.cc:460] Webserver started at http://127.14.51.126:42191/ using document root <none> and password file <none>
I20260806 06:41:38.362934 17040 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:38.362977 17040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:38.363153 17040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:38.364218 17040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "b7ac4c3db82a402b91efb88cddd4417d"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "c92e474c10f8968d9657c54de1a51259"
server_key_iv: "2be66c88005a4bd1c0d52923dc68b406"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.364574 17040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "b7ac4c3db82a402b91efb88cddd4417d"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "c92e474c10f8968d9657c54de1a51259"
server_key_iv: "2be66c88005a4bd1c0d52923dc68b406"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.365841 17040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:38.366534 17055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.366732 17040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:38.366817 17040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "b7ac4c3db82a402b91efb88cddd4417d"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "c92e474c10f8968d9657c54de1a51259"
server_key_iv: "2be66c88005a4bd1c0d52923dc68b406"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.366869 17040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:38.378294 17040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:38.380034 17040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:38.380138 17040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:38.383783 17040 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:43361
I20260806 06:41:38.383872 17111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:43361 every 8 connection(s)
I20260806 06:41:38.384184 17040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:38.385030 17112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:38.387279 17112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d: Bootstrap starting.
I20260806 06:41:38.387806 17112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:38.388083 17112 log.cc:824] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:38.388751 17112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d: No bootstrap required, opened a new log
I20260806 06:41:38.390209 17112 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } }
I20260806 06:41:38.390367 17112 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:38.390455 17112 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ac4c3db82a402b91efb88cddd4417d, State: Initialized, Role: FOLLOWER
I20260806 06:41:38.390607 17112 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } }
I20260806 06:41:38.390687 17112 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:38.390725 17112 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:38.390774 17112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:38.391337 17112 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } }
I20260806 06:41:38.391458 17112 leader_election.cc:302] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7ac4c3db82a402b91efb88cddd4417d; no voters:
I20260806 06:41:38.391633 17112 leader_election.cc:288] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:38.391682 17115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:38.391855 17115 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [term 1 LEADER]: Becoming Leader. State: Replica: b7ac4c3db82a402b91efb88cddd4417d, State: Running, Role: LEADER
I20260806 06:41:38.391870 17112 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:38.391985 17115 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } }
I20260806 06:41:38.392449 17117 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ac4c3db82a402b91efb88cddd4417d. Latest consensus state: current_term: 1 leader_uuid: "b7ac4c3db82a402b91efb88cddd4417d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } } }
I20260806 06:41:38.392529 17117 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:38.392944 17124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:38.392921 17116 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7ac4c3db82a402b91efb88cddd4417d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ac4c3db82a402b91efb88cddd4417d" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43361 } } }
I20260806 06:41:38.393028 17116 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:38.393440 17124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:38.393972 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17040
I20260806 06:41:38.394049 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:38.394198 14541 external_mini_cluster.cc:1428] Setting key e3046d663ad2bca7bc7def67cb8f3873
I20260806 06:41:38.394949 17124 catalog_manager.cc:1347] Generated new cluster ID: c50e028ec94347e39d5f2320d1b56cdf
I20260806 06:41:38.395021 17124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:38.401090 17124 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:38.401922 17124 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:38.407191 17124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7ac4c3db82a402b91efb88cddd4417d: Generated new TSK 0
I20260806 06:41:38.407349 17124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:38.409775 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.394808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.14.51.126:43361)
0806 06:41:38.399252 (+ 4444us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:38.399261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:38.399292 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:41:38.399397 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:38.399403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:38.399453 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:38.399580 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:38.400348 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.401691 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.402763 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.403394 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.403495 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.406278 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:38.406281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:38.406282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:38.406294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:38.408231 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.408613 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.408616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.408617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.408717 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.408938 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.408939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.408940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.409174 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:38.409331 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:38.409452 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:41:38.409601 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4397,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:38.434077 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:38.514117 17138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:38.514305 17138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:38.514328 17138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:38.515416 17138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:38.515450 17138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:38.515476 17138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:38.516791 17138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:38.516963 17138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:38.517134 17138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:38.521955 17144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.522003 17143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.522097 17138 server_base.cc:1034] running on GCE node
W20260806 06:41:38.522006 17146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.522379 17138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:38.522624 17138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:38.523764 17138 hybrid_clock.cc:648] HybridClock initialized: now 1785998498523742 us; error 45 us; skew 500 ppm
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:38.526253 17138 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:38.526682 17138 webserver.cc:460] Webserver started at http://127.14.51.65:39505/ using document root <none> and password file <none>
I20260806 06:41:38.526865 17138 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:38.526904 17138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:38.526994 17138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:38.528009 17138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "c9d5311fa1f949b9a17c0be1ebd16ee9"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "b0debb13b0f12522812f537e946661f2"
server_key_iv: "c4dece01771f375c056236ab632e387c"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.528379 17138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "c9d5311fa1f949b9a17c0be1ebd16ee9"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "b0debb13b0f12522812f537e946661f2"
server_key_iv: "c4dece01771f375c056236ab632e387c"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.529580 17138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:38.530263 17153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.530442 17138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:38.530540 17138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "c9d5311fa1f949b9a17c0be1ebd16ee9"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "b0debb13b0f12522812f537e946661f2"
server_key_iv: "c4dece01771f375c056236ab632e387c"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.530623 17138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:38.543298 17138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:38.544976 17138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:38.545138 17138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:38.545332 17138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:38.545614 17138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:38.545666 17138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.545703 17138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:38.545730 17138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.551460 17138 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:45197
I20260806 06:41:38.551515 17271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:45197 every 8 connection(s)
I20260806 06:41:38.551856 17138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:38.558666 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17138
I20260806 06:41:38.558785 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:38.559017 14541 external_mini_cluster.cc:1428] Setting key 9af491399adb0f08ab057954be4c4bd8
I20260806 06:41:38.561010 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.552519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43361 (local address 127.14.51.65:50977)
0806 06:41:38.552654 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:38.552696 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:38.553157 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:38.553414 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:38.553417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:38.553960 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:38.554123 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.554127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.554812 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.554813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.555650 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.555653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.556054 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.556055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.556178 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.556562 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:38.556570 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:38.557847 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:38.559684 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.559691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.559693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.559945 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.560165 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.560166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.560166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.560257 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.560569 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:38.560573 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:38.560674 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:41:38.560753 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:41:38.560871 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260806 06:41:38.561067 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.552552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:50977 (local address 127.14.51.126:43361)
0806 06:41:38.552595 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:38.552597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:38.552778 (+ 181us) server_negotiation.cc:407] Connection header received
0806 06:41:38.553258 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:38.553260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:38.553273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:38.553332 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:38.554227 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.554722 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.555748 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.555972 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.556049 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.557936 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:38.557938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:38.557938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:38.557945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:38.559600 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.560032 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.560033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.560033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.560097 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.560317 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.560318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.560318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.560499 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:38.560570 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:38.560817 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 06:41:38.560909 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:38.561816 17272 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43361
I20260806 06:41:38.561920 17272 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:38.562079 17272 heartbeater.cc:511] Master 127.14.51.126:43361 requested a full tablet report, sending...
I20260806 06:41:38.562569 17073 ts_manager.cc:194] Registered new tserver with Master: c9d5311fa1f949b9a17c0be1ebd16ee9 (127.14.51.65:45197)
I20260806 06:41:38.563272 17073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:50977
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:38.585201 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:38.663887 17277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:38.664072 17277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:38.664094 17277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:38.665148 17277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:38.665184 17277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:38.665208 17277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:38.666505 17277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:38.666666 17277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:38.666821 17277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:38.671564 17285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.671636 17277 server_base.cc:1034] running on GCE node
W20260806 06:41:38.671603 17282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.671595 17283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.672093 17277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:38.672350 17277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:38.673524 17277 hybrid_clock.cc:648] HybridClock initialized: now 1785998498673522 us; error 38 us; skew 500 ppm
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:38.676151 17277 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:38.676563 17277 webserver.cc:460] Webserver started at http://127.14.51.66:41897/ using document root <none> and password file <none>
I20260806 06:41:38.676802 17277 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:38.676879 17277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:38.677030 17277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:38.678071 17277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "67eb18c8127a47a28a1d15e868ed6472"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "d74a8551e0182e5fefefc2dc6f236bc5"
server_key_iv: "9fcb9b64b92f7c03c73458885f62d37b"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.678414 17277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "67eb18c8127a47a28a1d15e868ed6472"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "d74a8551e0182e5fefefc2dc6f236bc5"
server_key_iv: "9fcb9b64b92f7c03c73458885f62d37b"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.679797 17277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:41:38.680563 17292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.680751 17277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:38.680917 17277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "67eb18c8127a47a28a1d15e868ed6472"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "d74a8551e0182e5fefefc2dc6f236bc5"
server_key_iv: "9fcb9b64b92f7c03c73458885f62d37b"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.680991 17277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:38.736832 17277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:38.738790 17277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:38.738971 17277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:38.739298 17277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:38.739619 17277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:38.739672 17277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.739706 17277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:38.739732 17277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.745695 17277 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:35535
I20260806 06:41:38.745760 17409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:35535 every 8 connection(s)
I20260806 06:41:38.746104 17277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:38.749629 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17277
I20260806 06:41:38.749709 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:38.749895 14541 external_mini_cluster.cc:1428] Setting key fd60af7bca320475c5c5e8f6450941ef
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:38.755774 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.746858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:56719 (local address 127.14.51.126:43361)
0806 06:41:38.746922 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:38.746924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:38.747092 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:41:38.747623 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:38.747624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:38.747634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:38.747689 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:38.748558 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.749030 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.750216 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.750482 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.750544 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.752673 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:38.752675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:38.752675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:38.752683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:38.754325 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.754689 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.754690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.754691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.754751 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.755073 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.755074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.755075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.755250 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:38.755349 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:38.755601 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 06:41:38.755683 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:41:38.755995 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.746812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43361 (local address 127.14.51.66:56719)
0806 06:41:38.746966 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:38.747006 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:38.747538 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:38.747740 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:38.747743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:38.748290 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:38.748485 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.748489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.749108 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.749108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.750135 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.750138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.750567 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.750568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.750712 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.751252 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:38.751259 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:38.752588 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:38.754404 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.754410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.754411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.754614 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.754849 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.754850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.754850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.754990 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.755339 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:38.755345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:38.755468 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:41:38.755579 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 06:41:38.755734 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260806 06:41:38.756425 17410 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43361
I20260806 06:41:38.756515 17410 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:38.756703 17410 heartbeater.cc:511] Master 127.14.51.126:43361 requested a full tablet report, sending...
I20260806 06:41:38.757083 17073 ts_manager.cc:194] Registered new tserver with Master: 67eb18c8127a47a28a1d15e868ed6472 (127.14.51.66:35535)
I20260806 06:41:38.757449 17073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:56719
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:38.775858 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:38.858834 17415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:38.859179 17415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:38.859232 17415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:38.861202 17415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:38.861263 17415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:38.861308 17415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:38.863571 17415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:38.863860 17415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:38.864151 17415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:38.869722 17421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.869720 17420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:38.869800 17423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:38.870030 17415 server_base.cc:1034] running on GCE node
I20260806 06:41:38.870313 17415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:38.870596 17415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:38.871721 17415 hybrid_clock.cc:648] HybridClock initialized: now 1785998498871700 us; error 40 us; skew 500 ppm
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:38.875003 17415 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:38.875595 17415 webserver.cc:460] Webserver started at http://127.14.51.67:42655/ using document root <none> and password file <none>
I20260806 06:41:38.875866 17415 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:38.875933 17415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:38.876080 17415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:38.877521 17415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "ea044d3035c044eb9c96833303619596"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "8619946721b549cfa016d37b0cb27350"
server_key_iv: "599f9e87ae022e7e0cfc4408e0feb631"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.877981 17415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "ea044d3035c044eb9c96833303619596"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "8619946721b549cfa016d37b0cb27350"
server_key_iv: "599f9e87ae022e7e0cfc4408e0feb631"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.879895 17415 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 06:41:38.880688 17430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.880873 17415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:38.880959 17415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "ea044d3035c044eb9c96833303619596"
format_stamp: "Formatted at 2026-08-06 06:41:38 on dist-test-slave-1xrh"
server_key: "8619946721b549cfa016d37b0cb27350"
server_key_iv: "599f9e87ae022e7e0cfc4408e0feb631"
server_key_version: "encryptionkey@0"
I20260806 06:41:38.881024 17415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:38.890585 17415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:38.893218 17415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:38.893353 17415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:38.893575 17415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:38.893899 17415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:38.893944 17415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.893980 17415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:38.894009 17415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:38.899672 17415 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:36689
I20260806 06:41:38.899715 17547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:36689 every 8 connection(s)
I20260806 06:41:38.899998 17415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:38.900293 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17415
I20260806 06:41:38.900400 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:38.900609 14541 external_mini_cluster.cc:1428] Setting key ac33be4d0b9f63e58a3cf9512698597a
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998498, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:38.909126 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.900770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:41895 (local address 127.14.51.126:43361)
0806 06:41:38.900837 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:38.900840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:38.901065 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:41:38.901715 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:38.901716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:38.901728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:38.901801 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:38.902732 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.903339 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.904320 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.904527 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.904600 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.906532 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:38.906533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:38.906534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:38.906540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:38.907972 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.908393 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.908394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.908395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.908446 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:38.908650 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:38.908651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:38.908651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:38.908744 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:38.908792 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:38.909009 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:41:38.909065 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:41:38.909314 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:38.900766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43361 (local address 127.14.51.67:41895)
0806 06:41:38.900947 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:38.900991 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:38.901608 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:38.901865 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:38.901869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:38.902463 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:38.902637 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.902641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.903427 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.903428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.904219 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:38.904221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:38.904604 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:38.904605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:38.904733 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:38.905121 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:38.905129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:38.906452 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:38.908047 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.908053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.908054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.908328 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.908501 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:38.908502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:38.908502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:38.908596 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:38.908794 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:38.908798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:38.908901 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 06:41:38.908979 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:41:38.909102 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260806 06:41:38.909781 17548 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43361
I20260806 06:41:38.909868 17548 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:38.910039 17548 heartbeater.cc:511] Master 127.14.51.126:43361 requested a full tablet report, sending...
I20260806 06:41:38.910390 17073 ts_manager.cc:194] Registered new tserver with Master: ea044d3035c044eb9c96833303619596 (127.14.51.67:36689)
I20260806 06:41:38.910732 17073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:41895
I20260806 06:41:38.912182 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:41:38.924680 17135 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:38.923881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.14.51.126:43361)
0806 06:41:38.923953 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:38.923955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:38.924070 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:41:38.924110 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:38.924111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:38.924119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:38.924165 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:38.924622 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43306: BlockingRecv error: recv got EOF from 127.0.0.1:43306 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:41:38.926040 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17138
I20260806 06:41:38.934859 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17277
I20260806 06:41:38.939546 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17415
I20260806 06:41:38.943939 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17040
2026-08-06T06:41:38Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2934 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17560](info): set up 2 sockets
Aug 06 06:41:38 dist-test-slave-1xrh krb5kdc[17560](info): commencing operation
krb5kdc: starting...
W20260806 06:41:40.008874 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:40.123287 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38425
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:40.202630 17573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:40.202818 17573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:40.202839 17573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:40.204010 17573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:40.204052 17573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:40.204067 17573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:40.204116 17573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:40.204141 17573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:40.205621 17573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:40.205859 17573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:40.206081 17573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:40.210937 17578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:40.210896 17581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:40.210925 17579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:40.211272 17573 server_base.cc:1034] running on GCE node
I20260806 06:41:40.211522 17573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:40.211796 17573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:40.212955 17573 hybrid_clock.cc:648] HybridClock initialized: now 1785998500212916 us; error 55 us; skew 500 ppm
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998500, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:40.230583 17573 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:40.231022 17573 webserver.cc:460] Webserver started at http://127.14.51.126:38503/ using document root <none> and password file <none>
I20260806 06:41:40.231312 17573 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:40.231379 17573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:40.231516 17573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:40.232534 17573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "61740e6a81f84ffdb8a34a5cd8de460a"
format_stamp: "Formatted at 2026-08-06 06:41:40 on dist-test-slave-1xrh"
server_key: "9a492deb03b212143da206577b36067b"
server_key_iv: "9c80f2b4763446f8c9a61e025965f8ca"
server_key_version: "encryptionkey@0"
I20260806 06:41:40.232883 17573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "61740e6a81f84ffdb8a34a5cd8de460a"
format_stamp: "Formatted at 2026-08-06 06:41:40 on dist-test-slave-1xrh"
server_key: "9a492deb03b212143da206577b36067b"
server_key_iv: "9c80f2b4763446f8c9a61e025965f8ca"
server_key_version: "encryptionkey@0"
I20260806 06:41:40.234200 17573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:40.234884 17588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:40.235110 17573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:40.235221 17573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "61740e6a81f84ffdb8a34a5cd8de460a"
format_stamp: "Formatted at 2026-08-06 06:41:40 on dist-test-slave-1xrh"
server_key: "9a492deb03b212143da206577b36067b"
server_key_iv: "9c80f2b4763446f8c9a61e025965f8ca"
server_key_version: "encryptionkey@0"
I20260806 06:41:40.235276 17573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:40.262310 17573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:40.264674 17573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:40.264787 17573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:40.268836 17573 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38425
I20260806 06:41:40.268872 17643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38425 every 8 connection(s)
I20260806 06:41:40.269418 17573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:40.270084 17644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:40.272238 17644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a: Bootstrap starting.
I20260806 06:41:40.272805 17644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:40.273099 17644 log.cc:824] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:40.273778 17644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a: No bootstrap required, opened a new log
I20260806 06:41:40.275465 17644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } }
I20260806 06:41:40.275594 17644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:40.275638 17644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61740e6a81f84ffdb8a34a5cd8de460a, State: Initialized, Role: FOLLOWER
I20260806 06:41:40.275782 17644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } }
I20260806 06:41:40.275842 17644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:40.275862 17644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:40.275952 17644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:40.276793 17644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } }
I20260806 06:41:40.276912 17644 leader_election.cc:302] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61740e6a81f84ffdb8a34a5cd8de460a; no voters:
I20260806 06:41:40.277101 17644 leader_election.cc:288] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:40.277168 17647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:40.277359 17647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [term 1 LEADER]: Becoming Leader. State: Replica: 61740e6a81f84ffdb8a34a5cd8de460a, State: Running, Role: LEADER
I20260806 06:41:40.277403 17644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:40.277433 17647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } }
I20260806 06:41:40.277858 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17573
I20260806 06:41:40.277854 17649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61740e6a81f84ffdb8a34a5cd8de460a. Latest consensus state: current_term: 1 leader_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } } }
I20260806 06:41:40.277933 17649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:40.277968 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:40.278095 17648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61740e6a81f84ffdb8a34a5cd8de460a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38425 } } }
I20260806 06:41:40.278151 17648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:40.278160 14541 external_mini_cluster.cc:1428] Setting key b06307c12998383e17882c7d511c2c51
I20260806 06:41:40.278604 17655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:40.279162 17655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:40.280790 17655 catalog_manager.cc:1347] Generated new cluster ID: eaa3f5a87f6f466cae77a197971c4bf5
I20260806 06:41:40.280846 17655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:40.289849 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:40.278746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.14.51.126:38425)
0806 06:41:40.278931 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:40.278940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:40.278977 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:41:40.279096 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:40.279104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:40.279183 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:40.279323 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:40.280327 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:40.281724 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:40.282811 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:40.283473 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:40.283585 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:40.286310 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:40.286314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:40.286314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:40.286325 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:40.288280 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:40.288665 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:40.288668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:40.288668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:40.288782 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:40.288982 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:40.288983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:40.288983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:40.289280 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:40.289453 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:40.289519 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:41:40.289648 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260806 06:41:40.290668 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:41:40.299865 17655 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:40.300590 17655 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:40.303517 17655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61740e6a81f84ffdb8a34a5cd8de460a: Generated new TSK 0
I20260806 06:41:40.303683 17655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:40.309859 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:40.306188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.14.51.126:38425)
0806 06:41:40.306247 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:40.306249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:40.306370 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:41:40.306445 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:40.306445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:40.306453 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:40.306496 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:40.306885 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:40.307429 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:40.308472 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:40.308669 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:40.308718 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:40.309314 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:40.309315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:40.309315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:40.309526 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:40.309544 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:40.309552 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:40.309624 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:40.309719 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:41:40.309741 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:41:40.312170 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17573
2026-08-06T06:41:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1366 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17677](info): set up 2 sockets
Aug 06 06:41:40 dist-test-slave-1xrh krb5kdc[17677](info): commencing operation
krb5kdc: starting...
W20260806 06:41:41.374766 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:41.489116 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33483
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:41.567022 17692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:41.567333 17692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:41.567379 17692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:41.568562 17692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:41.568629 17692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:41.568652 17692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:41.568673 17692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:41.568722 17692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:41.570129 17692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:41.570386 17692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:41.570600 17692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:41.575101 17698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:41.575263 17700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:41.575222 17697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:41.575398 17692 server_base.cc:1034] running on GCE node
I20260806 06:41:41.575619 17692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:41.575852 17692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:41.576995 17692 hybrid_clock.cc:648] HybridClock initialized: now 1785998501576989 us; error 38 us; skew 500 ppm
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:41.579365 17692 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:41.579782 17692 webserver.cc:460] Webserver started at http://127.14.51.126:33853/ using document root <none> and password file <none>
I20260806 06:41:41.579957 17692 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:41.579994 17692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:41.580080 17692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:41.581146 17692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "57aa6d97ea2f4b33909a917ebedaa8df"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "32b88c51df64b65ff415c799be846388"
server_key_iv: "96315523a7eba10dcd7d6138d36ba428"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.581485 17692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "57aa6d97ea2f4b33909a917ebedaa8df"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "32b88c51df64b65ff415c799be846388"
server_key_iv: "96315523a7eba10dcd7d6138d36ba428"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.582793 17692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:41.583621 17709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.583792 17692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.583917 17692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "57aa6d97ea2f4b33909a917ebedaa8df"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "32b88c51df64b65ff415c799be846388"
server_key_iv: "96315523a7eba10dcd7d6138d36ba428"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.583992 17692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:41.603973 17692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:41.605826 17692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:41.605996 17692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:41.609968 17692 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:33483
I20260806 06:41:41.610003 17763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:33483 every 8 connection(s)
I20260806 06:41:41.610364 17692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:41.611568 17764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:41.613256 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17692
I20260806 06:41:41.613372 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:41.613555 14541 external_mini_cluster.cc:1428] Setting key 1892a67bf54e9c75de3fedb394ae49a2
I20260806 06:41:41.614271 17764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df: Bootstrap starting.
I20260806 06:41:41.614892 17764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:41.615283 17764 log.cc:824] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:41.615991 17764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df: No bootstrap required, opened a new log
I20260806 06:41:41.618185 17764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } }
I20260806 06:41:41.618387 17764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:41.618431 17764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57aa6d97ea2f4b33909a917ebedaa8df, State: Initialized, Role: FOLLOWER
I20260806 06:41:41.618613 17764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } }
I20260806 06:41:41.618747 17764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:41.618800 17764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:41.618861 17764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:41.619661 17764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } }
I20260806 06:41:41.619784 17764 leader_election.cc:302] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57aa6d97ea2f4b33909a917ebedaa8df; no voters:
I20260806 06:41:41.619952 17764 leader_election.cc:288] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:41.619992 17769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:41.620148 17769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [term 1 LEADER]: Becoming Leader. State: Replica: 57aa6d97ea2f4b33909a917ebedaa8df, State: Running, Role: LEADER
I20260806 06:41:41.620220 17764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:41.620235 17769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } }
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:41.620563 17769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57aa6d97ea2f4b33909a917ebedaa8df. Latest consensus state: current_term: 1 leader_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } } }
I20260806 06:41:41.620642 17769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:41.620613 17771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aa6d97ea2f4b33909a917ebedaa8df" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33483 } } }
I20260806 06:41:41.620983 17771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:41.621345 17776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:41.621965 17776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:41.623678 17776 catalog_manager.cc:1347] Generated new cluster ID: 36deaf49c73d4cc7b3a3429e1258edef
I20260806 06:41:41.623729 17776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:41.624810 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:41.614065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.14.51.126:33483)
0806 06:41:41.614249 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:41.614256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:41.614281 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:41:41.614715 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:41.614722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:41.614768 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:41.614877 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:41.615737 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.616667 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.617783 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.618165 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.618242 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:41.621118 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:41.621122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:41.621122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:41.621131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:41.623117 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.623530 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.623533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.623533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.623637 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.623910 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.623911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.623911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.624213 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:41.624333 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:41.624491 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:41:41.624615 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 06:41:41.641646 17776 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:41.642366 17776 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:41.648212 17776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57aa6d97ea2f4b33909a917ebedaa8df: Generated new TSK 0
I20260806 06:41:41.648381 17776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:41.677606 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:41.757354 17790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:41.757545 17790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:41.757567 17790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:41.758638 17790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:41.758677 17790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:41.758702 17790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:41.760110 17790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:41.760330 17790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:41.760550 17790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:41.765326 17795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:41.765326 17798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:41.765367 17796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:41.765687 17790 server_base.cc:1034] running on GCE node
I20260806 06:41:41.765842 17790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:41.766019 17790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:41.767295 17790 hybrid_clock.cc:648] HybridClock initialized: now 1785998501767285 us; error 35 us; skew 500 ppm
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:41.769682 17790 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:41.770152 17790 webserver.cc:460] Webserver started at http://127.14.51.65:33935/ using document root <none> and password file <none>
I20260806 06:41:41.770385 17790 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:41.770429 17790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:41.770578 17790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:41.771621 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "801bf34d133745f2b12b10d0300b76a3"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "68583f47227fff8044946057a2697954"
server_key_iv: "3aaf4428c57aa68f53c2d5f71c7fe3fc"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.772001 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "801bf34d133745f2b12b10d0300b76a3"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "68583f47227fff8044946057a2697954"
server_key_iv: "3aaf4428c57aa68f53c2d5f71c7fe3fc"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.773197 17790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:41.773837 17805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.774031 17790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:41.774111 17790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "801bf34d133745f2b12b10d0300b76a3"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "68583f47227fff8044946057a2697954"
server_key_iv: "3aaf4428c57aa68f53c2d5f71c7fe3fc"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.774163 17790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:41.784271 17790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:41.786433 17790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:41.786561 17790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:41.786756 17790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:41.787030 17790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:41.787062 17790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.787083 17790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:41.787096 17790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.792833 17790 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34597
I20260806 06:41:41.792917 17922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34597 every 8 connection(s)
I20260806 06:41:41.793123 17790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:41.801463 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:41.793749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:59959 (local address 127.14.51.126:33483)
0806 06:41:41.793788 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:41.793790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:41.794010 (+ 220us) server_negotiation.cc:407] Connection header received
0806 06:41:41.794471 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:41.794472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:41.794480 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:41.794532 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:41.795450 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.795945 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.796890 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.797092 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.797142 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:41.799022 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:41.799024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:41.799024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:41.799031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:41.800419 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.800731 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.800732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.800732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.800775 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.800957 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.800958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.800958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.801057 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:41.801127 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:41.801330 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 06:41:41.801392 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:41.801616 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:41.793724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33483 (local address 127.14.51.65:59959)
0806 06:41:41.793888 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:41.793927 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:41.794397 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:41.794592 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:41.794595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:41.795118 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:41.795373 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:41.795377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.796020 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.796021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:41.796820 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:41.796821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.797163 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.797163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:41.797254 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:41.797685 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:41.797692 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:41.798943 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:41.800479 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:41.800484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:41.800485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:41.800673 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:41.800826 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:41.800827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:41.800827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:41.800909 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:41.801126 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:41.801130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:41.801228 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:41:41.801303 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:41:41.801408 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260806 06:41:41.802088 17923 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33483
I20260806 06:41:41.802186 17923 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:41.802269 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17790
I20260806 06:41:41.802390 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:41.802387 17923 heartbeater.cc:511] Master 127.14.51.126:33483 requested a full tablet report, sending...
I20260806 06:41:41.802587 14541 external_mini_cluster.cc:1428] Setting key 4272156d0855d5aa6ebe4a7d8843537e
I20260806 06:41:41.802944 17726 ts_manager.cc:194] Registered new tserver with Master: 801bf34d133745f2b12b10d0300b76a3 (127.14.51.65:34597)
I20260806 06:41:41.803565 17726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:59959
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:41.825865 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:41.904525 17928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:41.904713 17928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:41.904736 17928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:41.905862 17928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:41.905911 17928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:41.905963 17928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:41.907419 17928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:41.907630 17928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:41.907833 17928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:41.912712 17934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:41.912758 17933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:41.912766 17928 server_base.cc:1034] running on GCE node
W20260806 06:41:41.912730 17936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:41.913110 17928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:41.913290 17928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:41.914424 17928 hybrid_clock.cc:648] HybridClock initialized: now 1785998501914406 us; error 34 us; skew 500 ppm
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:41.916877 17928 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:41.917309 17928 webserver.cc:460] Webserver started at http://127.14.51.66:40503/ using document root <none> and password file <none>
I20260806 06:41:41.917490 17928 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:41.917529 17928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:41.917615 17928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:41.918558 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "27a5ca062581410fb4d1b006fac622a4"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "27e7d378156cff190199fb79cf520260"
server_key_iv: "661f576dddfc3d6160c8b257f62dde14"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.918844 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "27a5ca062581410fb4d1b006fac622a4"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "27e7d378156cff190199fb79cf520260"
server_key_iv: "661f576dddfc3d6160c8b257f62dde14"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.920149 17928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:41.920879 17943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.921049 17928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.921132 17928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "27a5ca062581410fb4d1b006fac622a4"
format_stamp: "Formatted at 2026-08-06 06:41:41 on dist-test-slave-1xrh"
server_key: "27e7d378156cff190199fb79cf520260"
server_key_iv: "661f576dddfc3d6160c8b257f62dde14"
server_key_version: "encryptionkey@0"
I20260806 06:41:41.921185 17928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:41.929934 17928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:41.932951 17928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:41.933108 17928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:41.933310 17928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:41.933599 17928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:41.933652 17928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.933691 17928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:41.933718 17928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:41.939347 17928 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:38533
I20260806 06:41:41.939410 18060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:38533 every 8 connection(s)
I20260806 06:41:41.939723 17928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:41.940017 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 17928
I20260806 06:41:41.940128 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:41.940320 14541 external_mini_cluster.cc:1428] Setting key 0dcdf9523f46d5332bb3d153e578284a
Aug 06 06:41:41 dist-test-slave-1xrh krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998501, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:41:41.950132 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:41.940638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:39427 (local address 127.14.51.126:33483)
0806 06:41:41.940688 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:41.940691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:41.940811 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:41:41.941377 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:41.941378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:41.941391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:41.941434 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:41.942356 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.943151 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.944597 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.944800 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.944878 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:41.947258 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:41.947260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:41.947260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:41.947267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:41.948613 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.949057 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.949058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.949059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.949118 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:41.949367 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:41.949368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:41.949369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:41.949484 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:41.949557 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:41.949954 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 06:41:41.950039 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:41:41.950318 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:41.940424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33483 (local address 127.14.51.66:39427)
0806 06:41:41.940686 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:41.940725 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:41.941269 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:41.941513 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:41.941517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:41.942049 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:41.942222 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:41.942226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.943260 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.943261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:41.944511 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:41.944515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:41.944902 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:41.944904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:41.945082 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:41.945560 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:41.945570 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:41.947163 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:41.948711 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:41.948718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:41.948719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:41.948980 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:41.949191 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:41.949191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:41.949192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:41.949298 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:41.949547 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:41.949553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:41.949687 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 06:41:41.949927 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 06:41:41.950096 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":93,"threads_started":1}
I20260806 06:41:41.950811 18061 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33483
I20260806 06:41:41.950938 18061 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:41.951102 18061 heartbeater.cc:511] Master 127.14.51.126:33483 requested a full tablet report, sending...
I20260806 06:41:41.951493 17726 ts_manager.cc:194] Registered new tserver with Master: 27a5ca062581410fb4d1b006fac622a4 (127.14.51.66:38533)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
I20260806 06:41:41.951920 17726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:39427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:41.966105 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:42.045693 18066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:42.045934 18066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:42.045970 18066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:42.047151 18066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:42.047217 18066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:42.047262 18066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:42.048671 18066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:42.048942 18066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:42.049176 18066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:42.053853 18074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:42.053916 18072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:42.053866 18071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:42.054097 18066 server_base.cc:1034] running on GCE node
I20260806 06:41:42.054318 18066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:42.054553 18066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:42.055712 18066 hybrid_clock.cc:648] HybridClock initialized: now 1785998502055697 us; error 28 us; skew 500 ppm
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998502, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:42.058169 18066 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:42.058610 18066 webserver.cc:460] Webserver started at http://127.14.51.67:33829/ using document root <none> and password file <none>
I20260806 06:41:42.058830 18066 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:42.058892 18066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:42.059051 18066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:42.060038 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "0dab2eb3377e422c840b35afd7b31a38"
format_stamp: "Formatted at 2026-08-06 06:41:42 on dist-test-slave-1xrh"
server_key: "2160e0cebcd132f7afbf0e9ab953d7f1"
server_key_iv: "25dfe83e442aacd0fb86ff441dcd6214"
server_key_version: "encryptionkey@0"
I20260806 06:41:42.060425 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "0dab2eb3377e422c840b35afd7b31a38"
format_stamp: "Formatted at 2026-08-06 06:41:42 on dist-test-slave-1xrh"
server_key: "2160e0cebcd132f7afbf0e9ab953d7f1"
server_key_iv: "25dfe83e442aacd0fb86ff441dcd6214"
server_key_version: "encryptionkey@0"
I20260806 06:41:42.061626 18066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:42.062289 18081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:42.062477 18066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:42.062544 18066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "0dab2eb3377e422c840b35afd7b31a38"
format_stamp: "Formatted at 2026-08-06 06:41:42 on dist-test-slave-1xrh"
server_key: "2160e0cebcd132f7afbf0e9ab953d7f1"
server_key_iv: "25dfe83e442aacd0fb86ff441dcd6214"
server_key_version: "encryptionkey@0"
I20260806 06:41:42.062628 18066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:42.095528 18066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:42.097826 18066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:42.097965 18066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:42.098135 18066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:42.098452 18066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:42.098488 18066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:42.098510 18066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:42.098562 18066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:42.104254 18066 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:33749
I20260806 06:41:42.104351 18200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:33749 every 8 connection(s)
I20260806 06:41:42.104645 18066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998502, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:42.110960 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18066
I20260806 06:41:42.111078 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:42.111287 14541 external_mini_cluster.cc:1428] Setting key 0b4acae496fb18dd859524b09379fddb
I20260806 06:41:42.113461 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:42.105369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:49487 (local address 127.14.51.126:33483)
0806 06:41:42.105450 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:42.105453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:42.105625 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:41:42.106086 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:42.106087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:42.106097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:42.106159 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:42.107029 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.107594 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.108532 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.108716 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.108764 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:42.110788 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:42.110790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:42.110790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:42.110796 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:42.112367 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:42.112715 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:42.112716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:42.112716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:42.112764 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:42.112969 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:42.112969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:42.112969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:42.113073 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:42.113122 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:42.113339 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:41:42.113397 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:41:42.113616 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:42.105320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33483 (local address 127.14.51.67:49487)
0806 06:41:42.105511 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:42.105545 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:42.106003 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:42.106211 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:42.106214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:42.106766 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:42.106952 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:42.106956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.107672 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.107673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:42.108458 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:42.108460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.108784 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.108785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:42.108884 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:42.109375 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:42.109382 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:42.110700 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:42.112441 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:42.112446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:42.112447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:42.112651 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:42.112820 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:42.112821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:42.112821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:42.112915 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:42.113126 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:42.113130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:42.113232 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 06:41:42.113310 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:41:42.113432 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
I20260806 06:41:42.114074 18201 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33483
I20260806 06:41:42.114177 18201 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:42.114348 18201 heartbeater.cc:511] Master 127.14.51.126:33483 requested a full tablet report, sending...
I20260806 06:41:42.114701 17726 ts_manager.cc:194] Registered new tserver with Master: 0dab2eb3377e422c840b35afd7b31a38 (127.14.51.67:33749)
I20260806 06:41:42.115058 17726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:49487
I20260806 06:41:42.122936 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[17677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998502, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[17677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998502, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:42.145196 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:42.138498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.14.51.126:33483)
0806 06:41:42.138591 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:42.138593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:42.138656 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:41:42.138692 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:42.138693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:42.138702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:42.138751 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:42.139331 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.139813 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.140535 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:42.140718 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:42.140767 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:42.142806 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:42.142808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:42.142808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:42.142816 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:42.144027 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:42.144374 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:42.144376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:42.144376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:42.144432 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:42.144629 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:42.144630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:42.144631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:42.144772 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:42.144882 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:42.144987 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:41:42.145094 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 06:41:42.145430 17726 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34470
I20260806 06:41:42.145915 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17790
I20260806 06:41:42.152451 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17928
I20260806 06:41:42.157132 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18066
I20260806 06:41:42.161293 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 17692
2026-08-06T06:41:42Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1851 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[18214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[18214](info): set up 2 sockets
Aug 06 06:41:42 dist-test-slave-1xrh krb5kdc[18214](info): commencing operation
krb5kdc: starting...
W20260806 06:41:43.210206 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:43.325969 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:42579
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:42579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:43.410537 18228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:43.410769 18228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:43.410822 18228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:43.412189 18228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:43.412261 18228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:43.412286 18228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:43.412314 18228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:43.412338 18228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:43.413913 18228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:42579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:42579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:43.414218 18228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:43.414445 18228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:43.419414 18234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.419420 18233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.419488 18236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:43.419715 18228 server_base.cc:1034] running on GCE node
I20260806 06:41:43.419957 18228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:43.420230 18228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:43.421397 18228 hybrid_clock.cc:648] HybridClock initialized: now 1785998503421376 us; error 33 us; skew 500 ppm
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:43.423941 18228 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:43.424366 18228 webserver.cc:460] Webserver started at http://127.14.51.126:39515/ using document root <none> and password file <none>
I20260806 06:41:43.424578 18228 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:43.424643 18228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:43.424789 18228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:43.425802 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "0dff08bdc4894d17900cdb7ce33fb8c7"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "47296bca8382dae5a0619a433adfbfdb"
server_key_iv: "f8f11d670541f3db51c751366af0829b"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.426151 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "0dff08bdc4894d17900cdb7ce33fb8c7"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "47296bca8382dae5a0619a433adfbfdb"
server_key_iv: "f8f11d670541f3db51c751366af0829b"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.427488 18228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:43.428181 18244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.428368 18228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:43.428469 18228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "0dff08bdc4894d17900cdb7ce33fb8c7"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "47296bca8382dae5a0619a433adfbfdb"
server_key_iv: "f8f11d670541f3db51c751366af0829b"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.428544 18228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:43.438794 18228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:43.441021 18228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:43.441176 18228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:43.445000 18228 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:42579
I20260806 06:41:43.445031 18299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:42579 every 8 connection(s)
I20260806 06:41:43.445508 18228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:43.446171 18300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:43.448467 18300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7: Bootstrap starting.
I20260806 06:41:43.449060 18300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:43.449365 18300 log.cc:824] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:43.450107 18300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7: No bootstrap required, opened a new log
I20260806 06:41:43.451669 18300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } }
I20260806 06:41:43.451797 18300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:43.451819 18300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff08bdc4894d17900cdb7ce33fb8c7, State: Initialized, Role: FOLLOWER
I20260806 06:41:43.451977 18300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } }
I20260806 06:41:43.452054 18300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:43.452095 18300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:43.452163 18300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:43.452776 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18228
I20260806 06:41:43.452932 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:43.453111 14541 external_mini_cluster.cc:1428] Setting key 6d0341e0a9a8f0cf8a4bb06910f595f1
I20260806 06:41:43.453152 18300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } }
I20260806 06:41:43.453294 18300 leader_election.cc:302] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dff08bdc4894d17900cdb7ce33fb8c7; no voters:
I20260806 06:41:43.453454 18300 leader_election.cc:288] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:43.453485 18303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:43.453711 18300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:43.453750 18303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [term 1 LEADER]: Becoming Leader. State: Replica: 0dff08bdc4894d17900cdb7ce33fb8c7, State: Running, Role: LEADER
I20260806 06:41:43.453938 18303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } }
I20260806 06:41:43.454339 18304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } } }
I20260806 06:41:43.454473 18304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:43.454694 18307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dff08bdc4894d17900cdb7ce33fb8c7. Latest consensus state: current_term: 1 leader_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff08bdc4894d17900cdb7ce33fb8c7" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 42579 } } }
I20260806 06:41:43.454774 18307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:43.454970 18314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:43.455677 18314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:43.457199 18314 catalog_manager.cc:1347] Generated new cluster ID: b50f788154ed41fd92dd9a989d732e7d
I20260806 06:41:43.457252 18314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:43.464572 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.453616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.14.51.126:42579)
0806 06:41:43.453851 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.453864 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.454035 (+ 171us) server_negotiation.cc:407] Connection header received
0806 06:41:43.454195 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.454203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.454263 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.455475 (+ 1212us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.455590 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.456960 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.457867 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.458367 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.458510 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.461019 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:43.461023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:43.461023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:43.461034 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:43.462862 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.463338 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.463342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.463342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.463436 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.463669 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.463669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.463670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.463934 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:43.464125 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:43.464209 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:41:43.464376 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260806 06:41:43.486213 18314 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:43.486763 18314 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:43.491731 18314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dff08bdc4894d17900cdb7ce33fb8c7: Generated new TSK 0
I20260806 06:41:43.491905 18314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:43.520015 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:43.598906 18327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:43.599136 18327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:43.599193 18327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:43.600340 18327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:43.600405 18327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:43.600449 18327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:43.601946 18327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:43.602196 18327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:43.602413 18327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:41:43.607358 18327 server_base.cc:1034] running on GCE node
W20260806 06:41:43.607281 18335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.607316 18333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.607281 18332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:43.607717 18327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:43.607967 18327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:43.609143 18327 hybrid_clock.cc:648] HybridClock initialized: now 1785998503609141 us; error 39 us; skew 500 ppm
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:43.611822 18327 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:43.612171 18327 webserver.cc:460] Webserver started at http://127.14.51.65:43879/ using document root <none> and password file <none>
I20260806 06:41:43.612331 18327 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:43.612368 18327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:43.612459 18327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:43.613392 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "61c9a6ef48e1405dbd8d15e5f0ec02f6"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "004914ada44c1b8705bb010b7412693b"
server_key_iv: "b6e32e33f604c8c4d056c290302c0496"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.613684 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "61c9a6ef48e1405dbd8d15e5f0ec02f6"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "004914ada44c1b8705bb010b7412693b"
server_key_iv: "b6e32e33f604c8c4d056c290302c0496"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.614838 18327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:43.615579 18344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.615761 18327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.615847 18327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "61c9a6ef48e1405dbd8d15e5f0ec02f6"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "004914ada44c1b8705bb010b7412693b"
server_key_iv: "b6e32e33f604c8c4d056c290302c0496"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.615899 18327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:43.633868 18327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:43.636052 18327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:43.636157 18327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:43.636322 18327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:43.636567 18327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:43.636597 18327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.636618 18327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:43.636631 18327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.642095 18327 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:35495
I20260806 06:41:43.642179 18461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:35495 every 8 connection(s)
I20260806 06:41:43.642462 18327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:43.644644 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18327
I20260806 06:41:43.644757 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:43.644949 14541 external_mini_cluster.cc:1428] Setting key 2a633e878e6631ad2f912b215e384311
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:43.652113 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.643148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:47621 (local address 127.14.51.126:42579)
0806 06:41:43.643195 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.643197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.643417 (+ 220us) server_negotiation.cc:407] Connection header received
0806 06:41:43.644361 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.644362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.644379 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.644449 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.645157 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.645867 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.646872 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.647096 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.647215 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.649287 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:43.649289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:43.649289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:43.649296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:43.650679 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.651085 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.651087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.651087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.651179 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.651458 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.651459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.651460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.651578 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:43.651652 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:43.651934 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:41:43.652025 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:43.652273 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.643101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:42579 (local address 127.14.51.65:47621)
0806 06:41:43.643296 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:43.643329 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:43.644247 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:43.644515 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:43.644518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:43.644804 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:43.645056 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.645062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.645962 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.645963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.646785 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.646787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.647225 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.647226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.647357 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.647860 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:43.647868 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:43.649191 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:43.650775 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.650782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.650784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.650996 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.651250 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.651251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.651251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.651357 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.651643 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:43.651649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:43.651795 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 06:41:43.651902 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:41:43.652051 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
I20260806 06:41:43.652799 18462 heartbeater.cc:347] Connected to a master server at 127.14.51.126:42579
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
I20260806 06:41:43.652889 18462 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:43.653026 18462 heartbeater.cc:511] Master 127.14.51.126:42579 requested a full tablet report, sending...
I20260806 06:41:43.653532 18261 ts_manager.cc:194] Registered new tserver with Master: 61c9a6ef48e1405dbd8d15e5f0ec02f6 (127.14.51.65:35495)
I20260806 06:41:43.654268 18261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:47621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:43.670120 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:43.749858 18467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:43.750083 18467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:43.750129 18467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:43.751250 18467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:43.751305 18467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:43.751348 18467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:43.752768 18467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:43.753016 18467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:43.753218 18467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:43.758251 18472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.758246 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:43.758385 18467 server_base.cc:1034] running on GCE node
W20260806 06:41:43.758247 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:43.758623 18467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:43.758843 18467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:43.759994 18467 hybrid_clock.cc:648] HybridClock initialized: now 1785998503759991 us; error 36 us; skew 500 ppm
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:43.762387 18467 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:43.762830 18467 webserver.cc:460] Webserver started at http://127.14.51.66:37899/ using document root <none> and password file <none>
I20260806 06:41:43.763051 18467 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:43.763116 18467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:43.763288 18467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:43.764256 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "dbe9cbc5d3aa4d3eba3736d770d97cae"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "3ff2bdb0a76d747816f6732f4e4c9f84"
server_key_iv: "4962e0f97ae9a52f00720d2137b9a961"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.764611 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "dbe9cbc5d3aa4d3eba3736d770d97cae"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "3ff2bdb0a76d747816f6732f4e4c9f84"
server_key_iv: "4962e0f97ae9a52f00720d2137b9a961"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.765962 18467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:43.766641 18483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.766813 18467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:43.766909 18467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "dbe9cbc5d3aa4d3eba3736d770d97cae"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "3ff2bdb0a76d747816f6732f4e4c9f84"
server_key_iv: "4962e0f97ae9a52f00720d2137b9a961"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.766988 18467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:43.795773 18467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:43.798130 18467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:43.798288 18467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:43.798529 18467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:43.798846 18467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:43.798899 18467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.798934 18467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:43.798961 18467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.805084 18467 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:35791
I20260806 06:41:43.805135 18601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:35791 every 8 connection(s)
I20260806 06:41:43.805464 18467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:43.813963 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.806163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:56589 (local address 127.14.51.126:42579)
0806 06:41:43.806214 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.806217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.806443 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:41:43.807373 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.807374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.807385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.807458 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.807982 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.808474 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.809362 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.809528 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.809573 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.811568 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:43.811570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:43.811570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:43.811577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:43.812677 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.813064 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.813065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.813066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.813124 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.813359 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.813360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.813360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.813469 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:43.813537 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:43.813795 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:41:43.813878 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:43.814134 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.806151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:42579 (local address 127.14.51.66:56589)
0806 06:41:43.806316 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:43.806352 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:43.807281 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:43.807508 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:43.807511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:43.807749 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:43.807914 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.807918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.808554 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.808555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.809291 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.809293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.809597 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.809598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.809684 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.810147 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:43.810154 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:43.811473 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:43.812755 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.812761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.812763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.812971 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.813194 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.813195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.813195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.813292 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.813539 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:43.813546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:43.813667 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:41:43.813763 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:41:43.813914 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260806 06:41:43.814502 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18467
I20260806 06:41:43.814611 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:43.814649 18602 heartbeater.cc:347] Connected to a master server at 127.14.51.126:42579
I20260806 06:41:43.814740 18602 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:43.814810 14541 external_mini_cluster.cc:1428] Setting key 15d8979a8d475e523cdc59056466b5ae
I20260806 06:41:43.814914 18602 heartbeater.cc:511] Master 127.14.51.126:42579 requested a full tablet report, sending...
I20260806 06:41:43.815306 18260 ts_manager.cc:194] Registered new tserver with Master: dbe9cbc5d3aa4d3eba3736d770d97cae (127.14.51.66:35791)
I20260806 06:41:43.815829 18260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:56589
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:43.837739 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:43.915719 18607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:43.915946 18607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:43.915982 18607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:43.917160 18607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:43.917233 18607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:43.917279 18607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:43.918905 18607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:42579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:43.919186 18607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:43.919447 18607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:43.924357 18615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.924312 18613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:43.924332 18612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:43.924530 18607 server_base.cc:1034] running on GCE node
I20260806 06:41:43.924741 18607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:43.924983 18607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:43.926142 18607 hybrid_clock.cc:648] HybridClock initialized: now 1785998503926140 us; error 39 us; skew 500 ppm
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:43.928622 18607 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:43.929024 18607 webserver.cc:460] Webserver started at http://127.14.51.67:42067/ using document root <none> and password file <none>
I20260806 06:41:43.929239 18607 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:43.929301 18607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:43.929448 18607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:43.930459 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "30a468dcf29a47d1aac88a825e71aab8"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "8dd0480a769eaa3c125f3482d680c7ba"
server_key_iv: "40b3e11bc0f2bf193e469609f41c469c"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.930826 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "30a468dcf29a47d1aac88a825e71aab8"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "8dd0480a769eaa3c125f3482d680c7ba"
server_key_iv: "40b3e11bc0f2bf193e469609f41c469c"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.932343 18607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:41:43.933020 18622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.933218 18607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:43.933307 18607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "30a468dcf29a47d1aac88a825e71aab8"
format_stamp: "Formatted at 2026-08-06 06:41:43 on dist-test-slave-1xrh"
server_key: "8dd0480a769eaa3c125f3482d680c7ba"
server_key_iv: "40b3e11bc0f2bf193e469609f41c469c"
server_key_version: "encryptionkey@0"
I20260806 06:41:43.933357 18607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:43.948123 18607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:43.949738 18607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:43.949841 18607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:43.950068 18607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:43.950354 18607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:43.950385 18607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.950434 18607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:43.950462 18607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:43.956597 18607 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:45041
I20260806 06:41:43.956632 18742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:45041 every 8 connection(s)
I20260806 06:41:43.956949 18607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:43.961989 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18607
I20260806 06:41:43.962080 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:43.962301 14541 external_mini_cluster.cc:1428] Setting key a7fa62205cb4801638751ea8fcaaed90
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:43.965811 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.957661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:38387 (local address 127.14.51.126:42579)
0806 06:41:43.957723 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.957725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.957920 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:41:43.958851 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.958852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.958862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.958920 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.959545 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.960041 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.960964 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.961144 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.961197 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.963556 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:43.963558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:43.963558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:43.963564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:43.964709 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.965081 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.965081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.965082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.965131 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.965330 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.965331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.965331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.965427 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:43.965477 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:43.965688 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 06:41:43.965748 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:41:43.965996 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.957638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:42579 (local address 127.14.51.67:38387)
0806 06:41:43.957784 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:43.957822 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:43.958761 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:43.958970 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:43.958973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:43.959285 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:43.959470 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.959474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.960115 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.960115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.960883 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:43.960885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.961212 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.961212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:43.961305 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.961705 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:43.961712 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:43.963446 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:43.964778 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.964783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.964784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.964999 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.965186 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:43.965187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:43.965187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:43.965277 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:43.965479 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:43.965482 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:43.965582 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:41:43.965658 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:41:43.965767 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260806 06:41:43.966514 18743 heartbeater.cc:347] Connected to a master server at 127.14.51.126:42579
I20260806 06:41:43.966610 18743 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:43.966768 18743 heartbeater.cc:511] Master 127.14.51.126:42579 requested a full tablet report, sending...
I20260806 06:41:43.967167 18260 ts_manager.cc:194] Registered new tserver with Master: 30a468dcf29a47d1aac88a825e71aab8 (127.14.51.67:45041)
I20260806 06:41:43.967563 18260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:38387
I20260806 06:41:43.973944 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:43.996598 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.989143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.14.51.65:35495)
0806 06:41:43.989299 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.989309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.989319 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:43.989345 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.989347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.989361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.989429 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.989976 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.990666 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.991434 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.991876 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.991954 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:43.993990 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:43.993997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:43.993997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:43.994003 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:43.995608 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.995910 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.995912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.995912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.995966 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:43.996129 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:43.996130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:43.996130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:43.996250 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:43.996352 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:43.996417 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:41:43.996476 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
Aug 06 06:41:43 dist-test-slave-1xrh krb5kdc[18214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:44.004379 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:43.997916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.14.51.126:42579)
0806 06:41:43.997979 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:43.997981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:43.997991 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:43.998013 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:43.998014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:43.998029 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:43.998073 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:43.998493 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.998964 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.999636 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:43.999811 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:43.999862 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:44.002012 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:44.002014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:44.002014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:44.002021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:44.003235 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:44.003562 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:44.003563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:44.003564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:44.003622 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:44.003842 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:44.003843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:44.003843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:44.003992 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:44.004092 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:44.004200 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:41:44.004276 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 06:41:44.004607 18260 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54734
I20260806 06:41:44.004938 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18327
I20260806 06:41:44.010510 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18467
I20260806 06:41:44.016522 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18607
I20260806 06:41:44.020908 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18228
2026-08-06T06:41:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1859 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:44 dist-test-slave-1xrh krb5kdc[18755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:44 dist-test-slave-1xrh krb5kdc[18755](info): set up 2 sockets
Aug 06 06:41:44 dist-test-slave-1xrh krb5kdc[18755](info): commencing operation
krb5kdc: starting...
W20260806 06:41:45.070433 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:45.183636 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38313
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:45.263182 18768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:45.263368 18768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:45.263391 18768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:45.264539 18768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:45.264580 18768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:45.264595 18768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:45.264648 18768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:45.264689 18768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:45.266135 18768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:45.266383 18768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:45.266598 18768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:45.271289 18774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.271256 18773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.271256 18776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:45.271456 18768 server_base.cc:1034] running on GCE node
I20260806 06:41:45.271836 18768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:45.272115 18768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:45.273285 18768 hybrid_clock.cc:648] HybridClock initialized: now 1785998505273269 us; error 34 us; skew 500 ppm
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:45.279875 18768 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:45.280149 18768 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:45.280190 18768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:45.280277 18768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:45.281240 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "a9ab92a16449474b84c4456fd6231bda"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "90a2bbf37f491ec7f12759dae6aff180"
server_key_iv: "953c9eb7c5ebcc3e16ac7cf24338e745"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.281522 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "a9ab92a16449474b84c4456fd6231bda"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "90a2bbf37f491ec7f12759dae6aff180"
server_key_iv: "953c9eb7c5ebcc3e16ac7cf24338e745"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.282670 18768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:41:45.283322 18783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.283489 18768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.283595 18768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "a9ab92a16449474b84c4456fd6231bda"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "90a2bbf37f491ec7f12759dae6aff180"
server_key_iv: "953c9eb7c5ebcc3e16ac7cf24338e745"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.283674 18768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:45.303589 18768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:45.305570 18768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:45.305740 18768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:45.310247 18768 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38313
I20260806 06:41:45.310271 18837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38313 every 8 connection(s)
I20260806 06:41:45.310686 18768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:45.312093 18838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:45.315711 18838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: Bootstrap starting.
I20260806 06:41:45.316507 18838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:45.316839 18838 log.cc:824] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:45.317627 18838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: No bootstrap required, opened a new log
I20260806 06:41:45.318071 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18768
I20260806 06:41:45.318207 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:45.318421 14541 external_mini_cluster.cc:1428] Setting key ba8891d9556334eddb0d73f0cc85dbaa
I20260806 06:41:45.320919 18838 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } }
I20260806 06:41:45.321092 18838 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:45.321133 18838 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ab92a16449474b84c4456fd6231bda, State: Initialized, Role: FOLLOWER
I20260806 06:41:45.321259 18838 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } }
I20260806 06:41:45.321341 18838 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:45.321367 18838 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:45.321417 18838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:45.322022 18838 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } }
I20260806 06:41:45.322129 18838 leader_election.cc:302] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9ab92a16449474b84c4456fd6231bda; no voters:
I20260806 06:41:45.322283 18838 leader_election.cc:288] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:45.322350 18843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:45.322494 18843 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [term 1 LEADER]: Becoming Leader. State: Replica: a9ab92a16449474b84c4456fd6231bda, State: Running, Role: LEADER
I20260806 06:41:45.322542 18838 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:45.322582 18843 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } }
I20260806 06:41:45.322925 18843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9ab92a16449474b84c4456fd6231bda. Latest consensus state: current_term: 1 leader_uuid: "a9ab92a16449474b84c4456fd6231bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } } }
I20260806 06:41:45.323017 18843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:45.322935 18844 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9ab92a16449474b84c4456fd6231bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ab92a16449474b84c4456fd6231bda" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38313 } } }
I20260806 06:41:45.323194 18844 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:45.323292 18852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:45.323999 18852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:45.328521 18852 catalog_manager.cc:1347] Generated new cluster ID: 86ca9c018f9c4460b297bb48114d1648
I20260806 06:41:45.328579 18852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:45.331281 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.318990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.14.51.126:38313)
0806 06:41:45.319231 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.319241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.319616 (+ 375us) server_negotiation.cc:407] Connection header received
0806 06:41:45.320863 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.320873 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.320936 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.321087 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:45.321876 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.323342 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.324531 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.324958 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.325086 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.327810 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:45.327815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:45.327815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:45.327834 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:45.329555 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.329932 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.329935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.329935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.330034 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.330260 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.330261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.330261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.330517 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:45.330688 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.330795 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.331045 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260806 06:41:45.341168 18852 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:45.341717 18852 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:45.344413 18852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: Generated new TSK 0
I20260806 06:41:45.344580 18852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:45.371747 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:45.452607 18865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:45.452797 18865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:45.452839 18865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:45.453996 18865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:45.454035 18865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:45.454061 18865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:45.455624 18865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:45.455891 18865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:45.456108 18865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:45.460987 18870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.461033 18873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.461033 18871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:45.461223 18865 server_base.cc:1034] running on GCE node
I20260806 06:41:45.461495 18865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:45.461728 18865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:45.462882 18865 hybrid_clock.cc:648] HybridClock initialized: now 1785998505462865 us; error 38 us; skew 500 ppm
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:45.465436 18865 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:45.465827 18865 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:45.465896 18865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:45.466054 18865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:45.467053 18865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "ec78bb02a3354e599690fd41033af204"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "f92022a43d8878832b1526e29d42821c"
server_key_iv: "bcbcaa8bd0148b045523c052ea07b67f"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.467483 18865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "ec78bb02a3354e599690fd41033af204"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "f92022a43d8878832b1526e29d42821c"
server_key_iv: "bcbcaa8bd0148b045523c052ea07b67f"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.468700 18865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:45.469388 18879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.469542 18865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:45.469635 18865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "ec78bb02a3354e599690fd41033af204"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "f92022a43d8878832b1526e29d42821c"
server_key_iv: "bcbcaa8bd0148b045523c052ea07b67f"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.469717 18865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:45.493887 18865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:45.495780 18865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:45.495936 18865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:45.496161 18865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:45.496474 18865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:45.496531 18865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.496565 18865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:45.496592 18865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.502166 18865 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44225
I20260806 06:41:45.502204 18996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44225 every 8 connection(s)
I20260806 06:41:45.502444 18865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:45.506325 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 18865
I20260806 06:41:45.506471 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:45.506650 14541 external_mini_cluster.cc:1428] Setting key d30a088e17a252a9013f0cc8b768a836
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:45.511581 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.503079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:47073 (local address 127.14.51.126:38313)
0806 06:41:45.503178 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.503181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.503338 (+ 157us) server_negotiation.cc:407] Connection header received
0806 06:41:45.503752 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.503753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.503762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.503829 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:45.504857 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.505364 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.506287 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.506603 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.506682 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.508940 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:45.508941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:45.508942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:45.508949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:45.510220 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.510655 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.510656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.510656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.510712 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.510941 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.510942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.510942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.511054 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:45.511126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.511416 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.511501 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 06:41:45.511729 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.503049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38313 (local address 127.14.51.65:47073)
0806 06:41:45.503229 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.503262 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.503676 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.503897 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.503917 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.504154 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:45.504780 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.504785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.505444 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.505445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.506191 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.506193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.506686 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.506687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.506846 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.507425 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:45.507433 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:45.508828 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:45.510315 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.510322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.510324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.510573 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.510771 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.510772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.510773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.510876 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.511118 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:45.511155 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:45.511280 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.511388 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.511523 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 06:41:45.512239 18997 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38313
I20260806 06:41:45.512313 18997 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:45.512455 18997 heartbeater.cc:511] Master 127.14.51.126:38313 requested a full tablet report, sending...
I20260806 06:41:45.512974 18801 ts_manager.cc:194] Registered new tserver with Master: ec78bb02a3354e599690fd41033af204 (127.14.51.65:44225)
I20260806 06:41:45.513952 18801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:47073
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:45.531425 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:45.609999 19002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:45.610196 19002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:45.610219 19002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:45.611378 19002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:45.611416 19002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:45.611440 19002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:45.612988 19002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:45.613233 19002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:45.613435 19002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:45.618086 19007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.618183 19008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.618183 19010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:45.618278 19002 server_base.cc:1034] running on GCE node
I20260806 06:41:45.618492 19002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:45.618669 19002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:45.619800 19002 hybrid_clock.cc:648] HybridClock initialized: now 1785998505619789 us; error 26 us; skew 500 ppm
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:45.622309 19002 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:45.622695 19002 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:45.622745 19002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:45.622877 19002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:45.624070 19002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "50d012554f1e4fb29ed7c4cd87def5a1"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "4ab673ca7641ac299da6f428f355658a"
server_key_iv: "376589e61772b987cbe0f20048abc555"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.624421 19002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "50d012554f1e4fb29ed7c4cd87def5a1"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "4ab673ca7641ac299da6f428f355658a"
server_key_iv: "376589e61772b987cbe0f20048abc555"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.625644 19002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:45.626346 19017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.626544 19002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:45.626621 19002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "50d012554f1e4fb29ed7c4cd87def5a1"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "4ab673ca7641ac299da6f428f355658a"
server_key_iv: "376589e61772b987cbe0f20048abc555"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.626670 19002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:45.670200 19002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:45.672614 19002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:45.672796 19002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:45.673024 19002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:45.673352 19002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:45.673422 19002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.673457 19002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:45.673484 19002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.679229 19002 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:40139
I20260806 06:41:45.679261 19134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:40139 every 8 connection(s)
I20260806 06:41:45.679533 19002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:45.686187 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19002
I20260806 06:41:45.686285 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:45.686527 14541 external_mini_cluster.cc:1428] Setting key 609c59e05c6b8603b78cde02d97f4fa0
I20260806 06:41:45.689034 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.680286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44369 (local address 127.14.51.126:38313)
0806 06:41:45.680336 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.680339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.680571 (+ 232us) server_negotiation.cc:407] Connection header received
0806 06:41:45.681028 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.681029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.681040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.681091 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:45.681957 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.682495 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.683487 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.683665 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.683718 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.685727 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:45.685728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:45.685728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:45.685734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:45.687475 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.687959 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.687960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.687960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.688008 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.688301 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.688301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.688302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.688429 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:45.688510 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.688816 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.688904 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:45.689200 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.680287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38313 (local address 127.14.51.66:44369)
0806 06:41:45.680455 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.680490 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.680948 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.681160 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.681163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.681420 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:45.681886 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.681891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.682570 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.682571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.683417 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.683421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.683734 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.683735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.683841 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.684239 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:45.684246 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:45.685622 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:45.687548 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.687554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.687556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.687884 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.688067 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.688068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.688068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.688238 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.688540 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:45.688562 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:45.688688 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.688770 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.688910 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260806 06:41:45.689733 19135 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38313
I20260806 06:41:45.689823 19135 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:45.689990 19135 heartbeater.cc:511] Master 127.14.51.126:38313 requested a full tablet report, sending...
I20260806 06:41:45.690407 18801 ts_manager.cc:194] Registered new tserver with Master: 50d012554f1e4fb29ed7c4cd87def5a1 (127.14.51.66:40139)
I20260806 06:41:45.690804 18801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:44369
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:45.712271 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:45.794492 19140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:45.794672 19140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:45.794696 19140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:45.795816 19140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:45.795856 19140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:45.795883 19140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:45.797472 19140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:45.797713 19140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:45.797959 19140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:45.802767 19146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.802862 19148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:45.802918 19145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:45.803040 19140 server_base.cc:1034] running on GCE node
I20260806 06:41:45.803269 19140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:45.803488 19140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:45.804643 19140 hybrid_clock.cc:648] HybridClock initialized: now 1785998505804627 us; error 44 us; skew 500 ppm
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:45.807112 19140 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:45.807461 19140 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:45.807507 19140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:45.807595 19140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:45.808514 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "c00acbaa51704798bc57e0aac49c6dca"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "07a1d236afa29b6fef9a84c7beacabf8"
server_key_iv: "cc604772469c65f7d5fefce7a0b942bb"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.808856 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "c00acbaa51704798bc57e0aac49c6dca"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "07a1d236afa29b6fef9a84c7beacabf8"
server_key_iv: "cc604772469c65f7d5fefce7a0b942bb"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.810072 19140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:41:45.810724 19154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.810894 19140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:45.810962 19140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "c00acbaa51704798bc57e0aac49c6dca"
format_stamp: "Formatted at 2026-08-06 06:41:45 on dist-test-slave-1xrh"
server_key: "07a1d236afa29b6fef9a84c7beacabf8"
server_key_iv: "cc604772469c65f7d5fefce7a0b942bb"
server_key_version: "encryptionkey@0"
I20260806 06:41:45.811012 19140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:45.832255 19140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:45.834069 19140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:45.834216 19140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:45.834416 19140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:45.834702 19140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:45.834734 19140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.834774 19140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:45.834797 19140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:45.840794 19140 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37847
I20260806 06:41:45.840907 19271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37847 every 8 connection(s)
I20260806 06:41:45.841136 19140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:45.846892 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19140
I20260806 06:41:45.846994 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:45.847222 14541 external_mini_cluster.cc:1428] Setting key 2d8bf81c8588b145c5b0aeed948681d2
Aug 06 06:41:45 dist-test-slave-1xrh krb5kdc[18755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:45.850639 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.841821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:54641 (local address 127.14.51.126:38313)
0806 06:41:45.841879 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.841881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.842060 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:41:45.842547 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.842548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.842562 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.842622 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:45.843618 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.844174 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.845408 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.845627 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.845707 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.848032 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:45.848033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:45.848034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:45.848041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:45.849333 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.849841 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.849842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.849843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.849916 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.850170 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.850171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.850171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.850274 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:45.850328 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.850510 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.850562 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:41:45.850857 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.841813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38313 (local address 127.14.51.67:54641)
0806 06:41:45.841943 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.841983 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.842447 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.842687 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.842691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.842931 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:45.843512 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.843518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.844277 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.844278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.845281 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.845284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.845718 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.845719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.845837 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.846235 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:45.846242 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:45.847946 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:45.849430 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.849436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.849438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.849751 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.849983 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:45.849984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:45.849984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:45.850112 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:45.850328 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:45.850331 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:45.850429 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.850500 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.850642 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260806 06:41:45.851300 19272 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38313
I20260806 06:41:45.851418 19272 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:45.851563 19272 heartbeater.cc:511] Master 127.14.51.126:38313 requested a full tablet report, sending...
I20260806 06:41:45.851955 18801 ts_manager.cc:194] Registered new tserver with Master: c00acbaa51704798bc57e0aac49c6dca (127.14.51.67:37847)
I20260806 06:41:45.852311 18801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:54641
I20260806 06:41:45.858747 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:41:45.865408 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.859994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.14.51.126:38313)
0806 06:41:45.860047 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.860049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.860152 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:41:45.860242 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.860243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.860252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.860295 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:45.860852 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.861284 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.861918 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.862099 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.862163 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.863144 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:45.863145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:45.863146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:45.863152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:45.864229 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.864598 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.864599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.864600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.864655 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:45.864856 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:45.864857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:45.864857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:45.864992 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:45.865106 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.865212 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.865298 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:41:45.866731 18801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:45.868291 18801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:45.874503 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:40139 (local address 127.0.0.1:36900)
0806 06:41:45.870672 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.870681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.870744 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.871045 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.871046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.871047 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:45.871088 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.871089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872168 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.872169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.873545 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.873548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874196 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.874285 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874297 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.874362 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.874418 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":78,"thread_start_us":51,"threads_started":1}
I20260806 06:41:45.874668 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37847 (local address 127.0.0.1:58056)
0806 06:41:45.870455 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.870489 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.870564 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.870891 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.870894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.870896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:45.870959 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.870960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872277 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.872279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.873494 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.873497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874409 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.874486 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874490 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.874543 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.874589 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260806 06:41:45.874675 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.14.51.67:37847)
0806 06:41:45.870734 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.870737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.870748 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:45.870781 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.870783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.870800 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.870892 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:45.871050 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872185 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.873578 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874246 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874315 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874373 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.874563 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.874611 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":56,"threads_started":1}
I20260806 06:41:45.875020 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.14.51.66:40139)
0806 06:41:45.870616 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.870621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.870808 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:41:45.870868 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.870871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.870887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.870958 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:45.871199 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872049 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.873638 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874101 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874195 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874265 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.874609 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.874651 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
I20260806 06:41:45.875022 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44225 (local address 127.0.0.1:47214)
0806 06:41:45.870735 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.870744 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.870783 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.871222 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.871223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.871225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:45.871263 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.871264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872428 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.872429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.873381 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.873384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874482 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.874541 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874851 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.874909 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.874952 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":82,"threads_started":1}
I20260806 06:41:45.875048 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.870754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.14.51.65:44225)
0806 06:41:45.871009 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.871013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.871022 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:45.871066 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.871067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.871085 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.871225 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:45.871318 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.872330 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.873465 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.874325 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.874383 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.874434 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.874928 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.874969 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260806 06:41:45.875504 19204 tablet_service.cc:1467] Processing CreateTablet for tablet 27a338edb6984f899214f722c0180104 (DEFAULT_TABLE table=test-table [id=e62e8fcdd28d4ac9841c9d9b97f31bad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:45.875656 19066 tablet_service.cc:1467] Processing CreateTablet for tablet 27a338edb6984f899214f722c0180104 (DEFAULT_TABLE table=test-table [id=e62e8fcdd28d4ac9841c9d9b97f31bad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:45.875860 19204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27a338edb6984f899214f722c0180104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:45.875895 18927 tablet_service.cc:1467] Processing CreateTablet for tablet 27a338edb6984f899214f722c0180104 (DEFAULT_TABLE table=test-table [id=e62e8fcdd28d4ac9841c9d9b97f31bad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:45.876006 19066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27a338edb6984f899214f722c0180104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:45.876186 18927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27a338edb6984f899214f722c0180104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:45.878597 19289 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Bootstrap starting.
I20260806 06:41:45.879199 19290 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Bootstrap starting.
I20260806 06:41:45.879467 19289 tablet_bootstrap.cc:654] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:45.879686 19291 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: Bootstrap starting.
I20260806 06:41:45.880149 19290 tablet_bootstrap.cc:654] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:45.880201 19289 log.cc:824] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:45.880239 19291 tablet_bootstrap.cc:654] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:45.880664 19290 log.cc:824] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:45.880788 19291 log.cc:824] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:45.881083 19289 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: No bootstrap required, opened a new log
I20260806 06:41:45.881165 19289 ts_tablet_manager.cc:1397] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:41:45.881459 19291 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: No bootstrap required, opened a new log
I20260806 06:41:45.881511 19291 ts_tablet_manager.cc:1397] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:45.882550 19289 raft_consensus.cc:348] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.882706 19289 raft_consensus.cc:374] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:45.882743 19289 raft_consensus.cc:729] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d012554f1e4fb29ed7c4cd87def5a1, State: Initialized, Role: FOLLOWER
I20260806 06:41:45.882898 19289 consensus_queue.cc:260] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.882900 19291 raft_consensus.cc:348] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.883049 19291 raft_consensus.cc:374] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:45.883081 19291 raft_consensus.cc:729] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec78bb02a3354e599690fd41033af204, State: Initialized, Role: FOLLOWER
I20260806 06:41:45.883231 19289 ts_tablet_manager.cc:1428] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:41:45.883213 19291 consensus_queue.cc:260] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.883380 19135 heartbeater.cc:503] Master 127.14.51.126:38313 was elected leader, sending a full tablet report...
I20260806 06:41:45.883759 18997 heartbeater.cc:503] Master 127.14.51.126:38313 was elected leader, sending a full tablet report...
I20260806 06:41:45.883759 19291 ts_tablet_manager.cc:1428] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:41:45.883899 19290 tablet_bootstrap.cc:492] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: No bootstrap required, opened a new log
I20260806 06:41:45.883970 19290 ts_tablet_manager.cc:1397] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 06:41:45.885525 19290 raft_consensus.cc:348] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.885636 19290 raft_consensus.cc:374] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:45.885656 19290 raft_consensus.cc:729] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00acbaa51704798bc57e0aac49c6dca, State: Initialized, Role: FOLLOWER
I20260806 06:41:45.885778 19290 consensus_queue.cc:260] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.886023 19290 ts_tablet_manager.cc:1428] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:41:45.886071 19272 heartbeater.cc:503] Master 127.14.51.126:38313 was elected leader, sending a full tablet report...
I20260806 06:41:45.905529 19296 raft_consensus.cc:482] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:45.905612 19296 raft_consensus.cc:504] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.905938 19296 leader_election.cc:288] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d012554f1e4fb29ed7c4cd87def5a1 (127.14.51.66:40139), c00acbaa51704798bc57e0aac49c6dca (127.14.51.67:37847)
I20260806 06:41:45.909546 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.906201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:53235 (local address 127.14.51.67:37847)
0806 06:41:45.906242 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.906244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.906264 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:41:45.906288 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.906289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.906298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.906354 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:45.906554 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.907328 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.908680 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.909210 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.909282 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.909290 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.909452 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.909485 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:41:45.909556 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.906183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:45055 (local address 127.14.51.66:40139)
0806 06:41:45.906240 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.906242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.906370 (+ 128us) server_negotiation.cc:407] Connection header received
0806 06:41:45.906440 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.906442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.906452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.906508 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:45.906701 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.907381 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.908680 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.909207 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.909282 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.909290 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.909451 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.909485 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:41:45.909600 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.906097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37847 (local address 127.14.51.65:53235)
0806 06:41:45.906161 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.906174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.906222 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.906382 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.906382 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.906383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:45.906472 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.906474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.907401 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.907402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.908587 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.908590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.909290 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.909290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.909372 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.909373 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.909428 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.909469 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":4}
I20260806 06:41:45.909607 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.906097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:40139 (local address 127.14.51.65:45055)
0806 06:41:45.906299 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.906308 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:45.906361 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:45.906521 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:45.906522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:45.906524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:45.906594 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.906596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.907450 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.907451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.908588 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:45.908590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.909290 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.909291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:45.909367 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.909371 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:45.909439 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:41:45.909469 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":57,"thread_start_us":86,"threads_started":1}
I20260806 06:41:45.909934 19086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27a338edb6984f899214f722c0180104" candidate_uuid: "ec78bb02a3354e599690fd41033af204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" is_pre_election: true
I20260806 06:41:45.909966 19224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27a338edb6984f899214f722c0180104" candidate_uuid: "ec78bb02a3354e599690fd41033af204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c00acbaa51704798bc57e0aac49c6dca" is_pre_election: true
I20260806 06:41:45.910029 19086 raft_consensus.cc:2454] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec78bb02a3354e599690fd41033af204 in term 0.
I20260806 06:41:45.910104 19224 raft_consensus.cc:2454] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec78bb02a3354e599690fd41033af204 in term 0.
I20260806 06:41:45.910231 18880 leader_election.cc:302] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d012554f1e4fb29ed7c4cd87def5a1, ec78bb02a3354e599690fd41033af204; no voters:
I20260806 06:41:45.910353 19296 raft_consensus.cc:2781] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:45.910418 19296 raft_consensus.cc:482] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:45.910434 19296 raft_consensus.cc:3037] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:45.911005 19296 raft_consensus.cc:504] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.911163 19296 leader_election.cc:288] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [CANDIDATE]: Term 1 election: Requested vote from peers 50d012554f1e4fb29ed7c4cd87def5a1 (127.14.51.66:40139), c00acbaa51704798bc57e0aac49c6dca (127.14.51.67:37847)
I20260806 06:41:45.911278 19086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27a338edb6984f899214f722c0180104" candidate_uuid: "ec78bb02a3354e599690fd41033af204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d012554f1e4fb29ed7c4cd87def5a1"
I20260806 06:41:45.911294 19224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27a338edb6984f899214f722c0180104" candidate_uuid: "ec78bb02a3354e599690fd41033af204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c00acbaa51704798bc57e0aac49c6dca"
I20260806 06:41:45.911352 19086 raft_consensus.cc:3037] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:45.911382 19224 raft_consensus.cc:3037] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:45.912005 19086 raft_consensus.cc:2454] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec78bb02a3354e599690fd41033af204 in term 1.
I20260806 06:41:45.912015 19224 raft_consensus.cc:2454] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec78bb02a3354e599690fd41033af204 in term 1.
I20260806 06:41:45.912190 18882 leader_election.cc:302] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c00acbaa51704798bc57e0aac49c6dca, ec78bb02a3354e599690fd41033af204; no voters:
I20260806 06:41:45.912335 19296 raft_consensus.cc:2781] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:45.912500 19296 raft_consensus.cc:686] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 1 LEADER]: Becoming Leader. State: Replica: ec78bb02a3354e599690fd41033af204, State: Running, Role: LEADER
I20260806 06:41:45.912603 19296 consensus_queue.cc:237] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } }
I20260806 06:41:45.913360 18801 catalog_manager.cc:5495] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec78bb02a3354e599690fd41033af204 (127.14.51.65). New cstate: current_term: 1 leader_uuid: "ec78bb02a3354e599690fd41033af204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec78bb02a3354e599690fd41033af204" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44225 } health_report { overall_health: HEALTHY } } }
I20260806 06:41:45.920332 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.917917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.14.51.65:44225)
0806 06:41:45.917963 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:45.917965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:45.917975 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:45.918027 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:45.918028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:45.918043 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:45.918126 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:45.918288 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.918751 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.919577 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:45.919776 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:45.919859 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:45.920020 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:45.920110 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:45.920199 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:45.920227 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:41:45.920245 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:45.921842 19224 raft_consensus.cc:1261] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 1 FOLLOWER]: Refusing update from remote peer ec78bb02a3354e599690fd41033af204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:45.921842 19086 raft_consensus.cc:1261] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 1 FOLLOWER]: Refusing update from remote peer ec78bb02a3354e599690fd41033af204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:45.922216 19299 consensus_queue.cc:1035] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c00acbaa51704798bc57e0aac49c6dca" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:45.922362 19296 consensus_queue.cc:1035] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d012554f1e4fb29ed7c4cd87def5a1" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:45.924602 19302 mvcc.cc:205] Tried to move back new op lower bound from 7315449880253661184 to 7315449880218624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315449880253661184}]
I20260806 06:41:45.924858 19304 mvcc.cc:205] Tried to move back new op lower bound from 7315449880253661184 to 7315449880218624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315449880253661184}]
I20260806 06:41:45.925009 19303 mvcc.cc:205] Tried to move back new op lower bound from 7315449880253661184 to 7315449880218624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315449880253661184 or (T in {7315449880253661184})}]
I20260806 06:41:45.927086 18800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:45.927201 18800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:45.927946 18800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9ab92a16449474b84c4456fd6231bda: Sending DeleteTablet for 3 replicas of tablet 27a338edb6984f899214f722c0180104
I20260806 06:41:45.928292 19066 tablet_service.cc:1514] Processing DeleteTablet for tablet 27a338edb6984f899214f722c0180104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:45 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:36900
I20260806 06:41:45.928326 18927 tablet_service.cc:1514] Processing DeleteTablet for tablet 27a338edb6984f899214f722c0180104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:45 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:47214
I20260806 06:41:45.928386 19204 tablet_service.cc:1514] Processing DeleteTablet for tablet 27a338edb6984f899214f722c0180104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:45 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58056
I20260806 06:41:45.928514 19312 tablet_replica.cc:330] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: stopping tablet replica
I20260806 06:41:45.928622 19312 raft_consensus.cc:2229] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:45.928722 19312 raft_consensus.cc:2258] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:45.928728 19314 tablet_replica.cc:330] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204: stopping tablet replica
I20260806 06:41:45.928874 19314 raft_consensus.cc:2229] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:45.928983 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18865
I20260806 06:41:45.929023 19314 raft_consensus.cc:2258] T 27a338edb6984f899214f722c0180104 P ec78bb02a3354e599690fd41033af204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:45.930188 19312 ts_tablet_manager.cc:1905] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:45.930464 19313 tablet_replica.cc:330] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: stopping tablet replica
I20260806 06:41:45.930600 19313 raft_consensus.cc:2229] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:45.930729 19313 raft_consensus.cc:2258] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:45.931269 19313 ts_tablet_manager.cc:1905] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:45.931988 19312 ts_tablet_manager.cc:1918] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:45.932066 19312 log.cc:1196] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/27a338edb6984f899214f722c0180104
I20260806 06:41:45.932358 19312 ts_tablet_manager.cc:1939] T 27a338edb6984f899214f722c0180104 P 50d012554f1e4fb29ed7c4cd87def5a1: Deleting consensus metadata
I20260806 06:41:45.932894 18784 catalog_manager.cc:4841] TS 50d012554f1e4fb29ed7c4cd87def5a1 (127.14.51.66:40139): tablet 27a338edb6984f899214f722c0180104 (table test-table [id=e62e8fcdd28d4ac9841c9d9b97f31bad]) successfully deleted
I20260806 06:41:45.933022 19313 ts_tablet_manager.cc:1918] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:45.933110 19313 log.cc:1196] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/27a338edb6984f899214f722c0180104
I20260806 06:41:45.933339 19313 ts_tablet_manager.cc:1939] T 27a338edb6984f899214f722c0180104 P c00acbaa51704798bc57e0aac49c6dca: Deleting consensus metadata
I20260806 06:41:45.933754 18786 catalog_manager.cc:4841] TS c00acbaa51704798bc57e0aac49c6dca (127.14.51.67:37847): tablet 27a338edb6984f899214f722c0180104 (table test-table [id=e62e8fcdd28d4ac9841c9d9b97f31bad]) successfully deleted
W20260806 06:41:45.939532 18786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:44225 (error 108)
I20260806 06:41:45.939734 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19002
I20260806 06:41:45.940970 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:45.940803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44225 (local address 127.0.0.1:47230)
0806 06:41:45.940888 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:45.940930 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 06:41:45.941748 18786 catalog_manager.cc:4584] TS ec78bb02a3354e599690fd41033af204 (127.14.51.65:44225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27a338edb6984f899214f722c0180104: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44225: connect: Connection refused (error 111)
I20260806 06:41:45.945665 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19140
I20260806 06:41:45.950685 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 18768
2026-08-06T06:41:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1929 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T06:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:45Z Disabled control of system clock
I20260806 06:41:45.978518 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:41185
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:41185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:41:46.058594 19321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:46.058796 19321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:46.058817 19321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:46.059993 19321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:46.060034 19321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:46.060047 19321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:46.060060 19321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:46.060108 19321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:46.061409 19321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:41185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:46.061653 19321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:46.061937 19321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:46.066490 19326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.066506 19329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.066502 19327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.066695 19321 server_base.cc:1034] running on GCE node
I20260806 06:41:46.066990 19321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:46.067302 19321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:46.068461 19321 hybrid_clock.cc:648] HybridClock initialized: now 1785998506068438 us; error 43 us; skew 500 ppm
I20260806 06:41:46.069571 19321 webserver.cc:460] Webserver started at http://127.14.51.126:44055/ using document root <none> and password file <none>
I20260806 06:41:46.069804 19321 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:46.069870 19321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:46.070015 19321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:46.071051 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "98a749c223b9471983dda55c5384c895"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "5433ceb1b7cb4be2aa41c798b27df419"
server_key_iv: "ac3c362b097caad2138275f5b1d36279"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.071456 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "98a749c223b9471983dda55c5384c895"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "5433ceb1b7cb4be2aa41c798b27df419"
server_key_iv: "ac3c362b097caad2138275f5b1d36279"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.072664 19321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:46.073377 19335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.073557 19321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:46.073652 19321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "98a749c223b9471983dda55c5384c895"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "5433ceb1b7cb4be2aa41c798b27df419"
server_key_iv: "ac3c362b097caad2138275f5b1d36279"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.073734 19321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:46.076936 19321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:46.077191 19321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:46.077332 19321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:46.081058 19321 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:41185
I20260806 06:41:46.081092 19388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:41185 every 8 connection(s)
I20260806 06:41:46.081408 19321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:46.082104 19389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:46.082484 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19321
I20260806 06:41:46.082599 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:46.082864 14541 external_mini_cluster.cc:1428] Setting key 7e19e49b9de161c8806bedb29857de33
I20260806 06:41:46.085507 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.083548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.14.51.126:41185)
0806 06:41:46.083754 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.083760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.083782 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:41:46.084187 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.084193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.084230 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.084368 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.085095 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.085099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.085099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.085103 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.085151 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.085161 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.085237 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.085303 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.085357 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260806 06:41:46.086549 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: Bootstrap starting.
I20260806 06:41:46.087265 19389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:46.087581 19389 log.cc:824] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:46.088264 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: No bootstrap required, opened a new log
I20260806 06:41:46.089946 19389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } }
I20260806 06:41:46.090085 19389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:46.090134 19389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a749c223b9471983dda55c5384c895, State: Initialized, Role: FOLLOWER
I20260806 06:41:46.090270 19389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } }
I20260806 06:41:46.090349 19389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:46.090407 19389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:46.090461 19389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:46.091109 19389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } }
I20260806 06:41:46.091297 19389 leader_election.cc:302] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98a749c223b9471983dda55c5384c895; no voters:
I20260806 06:41:46.091490 19389 leader_election.cc:288] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:46.091723 19394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:46.091766 19389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:46.091902 19394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [term 1 LEADER]: Becoming Leader. State: Replica: 98a749c223b9471983dda55c5384c895, State: Running, Role: LEADER
I20260806 06:41:46.091987 19394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } }
I20260806 06:41:46.092388 19397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98a749c223b9471983dda55c5384c895. Latest consensus state: current_term: 1 leader_uuid: "98a749c223b9471983dda55c5384c895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } } }
I20260806 06:41:46.092391 19396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98a749c223b9471983dda55c5384c895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a749c223b9471983dda55c5384c895" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41185 } } }
I20260806 06:41:46.092489 19397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:46.092525 19396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:46.092684 19404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:46.093163 19404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:46.094668 19404 catalog_manager.cc:1347] Generated new cluster ID: a7f12922c1a644e6ab6a070ec047960e
I20260806 06:41:46.094720 19404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:46.101437 19404 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:46.101598 19404 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:46.104588 19404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: Generated new TSK 0
I20260806 06:41:46.104737 19404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:46.111500 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--builtin_ntp_servers=127.14.51.84:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:41:46.190007 19414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:46.190208 19414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:46.190230 19414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:46.191443 19414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:46.191556 19414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:46.191628 19414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:46.193186 19414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:46.193456 19414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:46.193781 19414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:46.199528 19419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.199539 19420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.200021 19414 server_base.cc:1034] running on GCE node
W20260806 06:41:46.200235 19422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.200470 19414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:46.200753 19414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:46.201946 19414 hybrid_clock.cc:648] HybridClock initialized: now 1785998506201921 us; error 39 us; skew 500 ppm
I20260806 06:41:46.203406 19414 webserver.cc:460] Webserver started at http://127.14.51.65:44307/ using document root <none> and password file <none>
I20260806 06:41:46.203681 19414 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:46.203747 19414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:46.203905 19414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:46.205291 19414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "a150ed1ab281453281e7f1604a6e1870"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "1b8a6b38133c1b52405479a7210146e1"
server_key_iv: "bde522164f993ca2e7072f5a7dbd6b99"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.205718 19414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "a150ed1ab281453281e7f1604a6e1870"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "1b8a6b38133c1b52405479a7210146e1"
server_key_iv: "bde522164f993ca2e7072f5a7dbd6b99"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.207255 19414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 06:41:46.208083 19428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.208277 19414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:46.208371 19414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "a150ed1ab281453281e7f1604a6e1870"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "1b8a6b38133c1b52405479a7210146e1"
server_key_iv: "bde522164f993ca2e7072f5a7dbd6b99"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.208437 19414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:46.212878 19414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:46.213146 19414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:46.213265 19414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:46.213465 19414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:46.213764 19414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:46.213819 19414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.213871 19414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:46.213891 19414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.221096 19414 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:40027
I20260806 06:41:46.221127 19544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:40027 every 8 connection(s)
I20260806 06:41:46.221483 19414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:46.224529 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.222190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:41113 (local address 127.14.51.126:41185)
0806 06:41:46.222256 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.222258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.222493 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:41:46.222924 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.222925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.222931 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.223009 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.224274 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.224276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.224276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.224281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.224297 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.224308 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.224371 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.224440 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.224454 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:41:46.224731 19547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.222160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41185 (local address 127.14.51.65:41113)
0806 06:41:46.222363 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.222410 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.222842 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.223056 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.223059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.223067 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.224124 (+ 1057us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.224125 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.224152 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.224178 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.224376 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.224380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.224382 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.224440 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.224485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260806 06:41:46.225211 19545 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41185
I20260806 06:41:46.225313 19545 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:46.225455 19545 heartbeater.cc:511] Master 127.14.51.126:41185 requested a full tablet report, sending...
I20260806 06:41:46.225981 19352 ts_manager.cc:194] Registered new tserver with Master: a150ed1ab281453281e7f1604a6e1870 (127.14.51.65:40027)
I20260806 06:41:46.226070 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19414
I20260806 06:41:46.226163 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:46.226330 14541 external_mini_cluster.cc:1428] Setting key 31a04112391631786a7e538d0b2b6ccb
I20260806 06:41:46.227329 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--builtin_ntp_servers=127.14.51.84:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:41:46.307621 19548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:46.307802 19548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:46.307826 19548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:46.308971 19548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:46.309011 19548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:46.309037 19548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:46.310462 19548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:46.310688 19548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:46.310961 19548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:46.316104 19553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.316124 19554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.316124 19556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.316313 19548 server_base.cc:1034] running on GCE node
I20260806 06:41:46.316509 19548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:46.316733 19548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:46.317919 19548 hybrid_clock.cc:648] HybridClock initialized: now 1785998506317912 us; error 36 us; skew 500 ppm
I20260806 06:41:46.319298 19548 webserver.cc:460] Webserver started at http://127.14.51.66:42617/ using document root <none> and password file <none>
I20260806 06:41:46.319563 19548 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:46.319636 19548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:46.319780 19548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:46.320798 19548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "aa2b531682ea49ae9493907554834aee"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "68694d5a19450b6a973f40ef50c78430"
server_key_iv: "00fd798aa4887c9d00d92106229970f2"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.321152 19548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "aa2b531682ea49ae9493907554834aee"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "68694d5a19450b6a973f40ef50c78430"
server_key_iv: "00fd798aa4887c9d00d92106229970f2"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.322700 19548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:46.323472 19562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.323642 19548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.323737 19548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "aa2b531682ea49ae9493907554834aee"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "68694d5a19450b6a973f40ef50c78430"
server_key_iv: "00fd798aa4887c9d00d92106229970f2"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.323824 19548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:46.327111 19548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:46.327390 19548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:46.327520 19548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:46.327724 19548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:46.328022 19548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:46.328075 19548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.328106 19548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:46.328132 19548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.333951 19548 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:36295
I20260806 06:41:46.333989 19680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:36295 every 8 connection(s)
I20260806 06:41:46.334299 19548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:46.337311 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.334999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:52291 (local address 127.14.51.126:41185)
0806 06:41:46.335049 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.335051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.335317 (+ 266us) server_negotiation.cc:407] Connection header received
0806 06:41:46.335784 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.335786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.335792 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.335838 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.337039 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.337041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.337042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.337047 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.337060 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.337064 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.337123 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.337216 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.337231 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:41:46.337491 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.334971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41185 (local address 127.14.51.66:52291)
0806 06:41:46.335181 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.335225 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.335675 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.335928 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.335933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.335948 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.336891 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.336892 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.336918 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.336943 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.337125 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.337130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.337132 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.337227 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.337286 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260806 06:41:46.338074 19681 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41185
I20260806 06:41:46.338148 19681 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:46.338308 19681 heartbeater.cc:511] Master 127.14.51.126:41185 requested a full tablet report, sending...
I20260806 06:41:46.338660 19352 ts_manager.cc:194] Registered new tserver with Master: aa2b531682ea49ae9493907554834aee (127.14.51.66:36295)
I20260806 06:41:46.341774 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19548
I20260806 06:41:46.341856 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:46.341986 14541 external_mini_cluster.cc:1428] Setting key 42436770336f2140bd156ac57aedae1a
I20260806 06:41:46.343004 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--builtin_ntp_servers=127.14.51.84:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:41:46.423542 19684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:46.423745 19684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:46.423767 19684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:46.424921 19684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:46.424960 19684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:46.424985 19684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:46.426388 19684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:46.426612 19684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:46.426891 19684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:46.431931 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.431910 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.432017 19684 server_base.cc:1034] running on GCE node
W20260806 06:41:46.431891 19692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.432325 19684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:46.432572 19684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:46.433740 19684 hybrid_clock.cc:648] HybridClock initialized: now 1785998506433726 us; error 45 us; skew 500 ppm
I20260806 06:41:46.435062 19684 webserver.cc:460] Webserver started at http://127.14.51.67:39519/ using document root <none> and password file <none>
I20260806 06:41:46.435323 19684 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:46.435400 19684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:46.435555 19684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:46.436587 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "d2650d2eb870482cbb8fd43b564a7489"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "6a81c60edaae05620283757378392713"
server_key_iv: "ed54909713fb30df767793b51914ead6"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.436941 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "d2650d2eb870482cbb8fd43b564a7489"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "6a81c60edaae05620283757378392713"
server_key_iv: "ed54909713fb30df767793b51914ead6"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.438373 19684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:46.439193 19698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.439349 19684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:46.439448 19684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "d2650d2eb870482cbb8fd43b564a7489"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "6a81c60edaae05620283757378392713"
server_key_iv: "ed54909713fb30df767793b51914ead6"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.439536 19684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:46.442826 19684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:46.443078 19684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:46.443244 19684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:46.443452 19684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:46.443773 19684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:46.443831 19684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.443871 19684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:46.443897 19684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.449599 19684 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:41171
I20260806 06:41:46.449630 19814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:41171 every 8 connection(s)
I20260806 06:41:46.449923 19684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:46.452533 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.450595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:41201 (local address 127.14.51.126:41185)
0806 06:41:46.450636 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.450638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.450825 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:41:46.451296 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.451297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.451302 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.451351 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.452353 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.452354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.452354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.452357 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.452369 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.452373 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.452423 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.452472 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.452483 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:46.452726 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.450560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41185 (local address 127.14.51.67:41201)
0806 06:41:46.450715 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.450744 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.451210 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.451397 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.451400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.451407 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.452227 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.452228 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.452252 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.452276 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.452423 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.452426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.452428 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.452471 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.452514 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260806 06:41:46.453235 19815 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41185
I20260806 06:41:46.453306 19815 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:46.453464 19815 heartbeater.cc:511] Master 127.14.51.126:41185 requested a full tablet report, sending...
I20260806 06:41:46.453832 19352 ts_manager.cc:194] Registered new tserver with Master: d2650d2eb870482cbb8fd43b564a7489 (127.14.51.67:41171)
I20260806 06:41:46.457410 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19684
I20260806 06:41:46.457511 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:46.457693 14541 external_mini_cluster.cc:1428] Setting key 40abec24f0842f4828a95f5952130d39
I20260806 06:41:46.458546 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:41:46.460841 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.459907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.14.51.126:41185)
0806 06:41:46.459962 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.459964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.460104 (+ 140us) server_negotiation.cc:407] Connection header received
0806 06:41:46.460151 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.460152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.460155 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.460194 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.460660 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.460661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.460661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.460682 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.460693 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.460696 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.460738 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.460787 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.460797 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:41:46.462440 19352 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:46.464010 19352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:46.467667 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:40027 (local address 127.0.0.1:36400)
0806 06:41:46.466302 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.466340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.466396 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.466685 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.466686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.466689 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.467195 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.467196 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.467211 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.467234 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.467495 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.467501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.467504 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.467589 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.467604 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":63,"threads_started":1}
I20260806 06:41:46.467715 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.14.51.65:40027)
0806 06:41:46.466548 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.466551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.466559 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:46.466580 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.466582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.466589 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.466670 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.467323 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.467331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.467331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.467336 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.467372 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.467403 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.467469 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.467612 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.467627 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":88,"threads_started":1}
I20260806 06:41:46.467973 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:41171 (local address 127.0.0.1:53082)
0806 06:41:46.466313 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.466340 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.466396 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.467223 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.467223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.467226 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.467628 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.467629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.467638 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.467652 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.467877 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.467878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.467878 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.467924 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.467934 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260806 06:41:46.468025 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.14.51.67:41171)
0806 06:41:46.467014 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.467017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.467024 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:46.467067 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.467069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.467077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.467220 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.467738 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.467744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.467744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.467750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.467784 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.467809 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.467877 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.467929 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.467943 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":71,"threads_started":1}
I20260806 06:41:46.468564 19477 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 (DEFAULT_TABLE table=test-table [id=4ce684d92e5245d885452d446d367903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:46.468592 19746 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 (DEFAULT_TABLE table=test-table [id=4ce684d92e5245d885452d446d367903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:46.468887 19746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd5cefe7a284fbd9d69aef8e86909f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:46.468894 19477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd5cefe7a284fbd9d69aef8e86909f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:46.471504 19829 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Bootstrap starting.
I20260806 06:41:46.471547 19831 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: Bootstrap starting.
I20260806 06:41:46.472163 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:36295 (local address 127.0.0.1:59624)
0806 06:41:46.466321 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.466348 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.466397 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.466654 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.466660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.466665 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.471745 (+ 5080us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.471745 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.471752 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.471762 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.472040 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.472041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.472041 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.472092 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.472105 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260806 06:41:46.472259 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.466306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.14.51.66:36295)
0806 06:41:46.466524 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.466527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.466534 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:46.466555 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.466557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.466564 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.466640 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.471864 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.471872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.471873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.471877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.471919 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.471944 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.471987 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.472131 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.472149 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260806 06:41:46.472343 19831 tablet_bootstrap.cc:654] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:46.472586 19829 tablet_bootstrap.cc:654] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:46.472669 19831 log.cc:824] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:46.472882 19829 log.cc:824] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:46.472894 19612 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 (DEFAULT_TABLE table=test-table [id=4ce684d92e5245d885452d446d367903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:46.473266 19612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd5cefe7a284fbd9d69aef8e86909f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:46.473520 19831 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: No bootstrap required, opened a new log
I20260806 06:41:46.473609 19831 ts_tablet_manager.cc:1397] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:46.476241 19831 raft_consensus.cc:348] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.476487 19831 raft_consensus.cc:374] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:46.476529 19831 raft_consensus.cc:729] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a150ed1ab281453281e7f1604a6e1870, State: Initialized, Role: FOLLOWER
I20260806 06:41:46.476625 19834 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Bootstrap starting.
I20260806 06:41:46.476660 19831 consensus_queue.cc:260] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.476940 19831 ts_tablet_manager.cc:1428] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:41:46.476984 19545 heartbeater.cc:503] Master 127.14.51.126:41185 was elected leader, sending a full tablet report...
I20260806 06:41:46.477349 19829 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: No bootstrap required, opened a new log
I20260806 06:41:46.477389 19834 tablet_bootstrap.cc:654] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:46.477433 19829 ts_tablet_manager.cc:1397] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 06:41:46.477727 19834 log.cc:824] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:46.478430 19834 tablet_bootstrap.cc:492] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: No bootstrap required, opened a new log
I20260806 06:41:46.478480 19834 ts_tablet_manager.cc:1397] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:46.479717 19829 raft_consensus.cc:348] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.479857 19829 raft_consensus.cc:374] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:46.479903 19829 raft_consensus.cc:729] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2650d2eb870482cbb8fd43b564a7489, State: Initialized, Role: FOLLOWER
I20260806 06:41:46.480011 19829 consensus_queue.cc:260] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.480274 19829 ts_tablet_manager.cc:1428] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:41:46.480295 19815 heartbeater.cc:503] Master 127.14.51.126:41185 was elected leader, sending a full tablet report...
I20260806 06:41:46.480386 19834 raft_consensus.cc:348] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.480556 19834 raft_consensus.cc:374] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:46.480608 19834 raft_consensus.cc:729] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2b531682ea49ae9493907554834aee, State: Initialized, Role: FOLLOWER
I20260806 06:41:46.480726 19834 consensus_queue.cc:260] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.480986 19834 ts_tablet_manager.cc:1428] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:41:46.481024 19681 heartbeater.cc:503] Master 127.14.51.126:41185 was elected leader, sending a full tablet report...
W20260806 06:41:46.584859 19682 tablet.cc:2369] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:46.652055 19837 raft_consensus.cc:482] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:46.652186 19837 raft_consensus.cc:504] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.652582 19837 leader_election.cc:288] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2b531682ea49ae9493907554834aee (127.14.51.66:36295), a150ed1ab281453281e7f1604a6e1870 (127.14.51.65:40027)
I20260806 06:41:46.654042 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.652755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:40027 (local address 127.14.51.67:45433)
0806 06:41:46.652793 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.652806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.652849 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.653111 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.653112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.653114 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.653616 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.653617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.653626 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.653641 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.653882 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.653885 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.653886 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.653946 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.653961 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 06:41:46.654084 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.652912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:45433 (local address 127.14.51.65:40027)
0806 06:41:46.652959 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.652961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.652994 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:41:46.653016 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.653017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.653021 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.653103 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.653744 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.653746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.653746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.653765 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.653788 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.653793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.653858 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.653963 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.653992 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:46.654163 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.652806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:45917 (local address 127.14.51.66:36295)
0806 06:41:46.652871 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.652873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.653047 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:41:46.653135 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.653137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.653145 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.653204 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.653913 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.653914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.653915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.653920 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.653938 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.653944 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.654002 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.654071 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.654086 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:41:46.654343 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.652756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:36295 (local address 127.14.51.67:45917)
0806 06:41:46.652938 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.652956 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.653061 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.653221 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.653222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.653224 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:46.653806 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:41:46.653806 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:46.653813 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:41:46.653825 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:46.654011 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:46.654011 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:46.654012 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:46.654065 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:41:46.654077 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":21,"thread_start_us":95,"threads_started":1}
I20260806 06:41:46.654393 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd5cefe7a284fbd9d69aef8e86909f1" candidate_uuid: "d2650d2eb870482cbb8fd43b564a7489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a150ed1ab281453281e7f1604a6e1870" is_pre_election: true
I20260806 06:41:46.654526 19497 raft_consensus.cc:2454] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2650d2eb870482cbb8fd43b564a7489 in term 0.
I20260806 06:41:46.654691 19634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd5cefe7a284fbd9d69aef8e86909f1" candidate_uuid: "d2650d2eb870482cbb8fd43b564a7489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2b531682ea49ae9493907554834aee" is_pre_election: true
I20260806 06:41:46.654788 19702 leader_election.cc:302] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a150ed1ab281453281e7f1604a6e1870, d2650d2eb870482cbb8fd43b564a7489; no voters:
I20260806 06:41:46.654841 19634 raft_consensus.cc:2454] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2650d2eb870482cbb8fd43b564a7489 in term 0.
I20260806 06:41:46.654942 19837 raft_consensus.cc:2781] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:46.655004 19837 raft_consensus.cc:482] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:46.655028 19837 raft_consensus.cc:3037] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:46.655754 19837 raft_consensus.cc:504] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.655889 19837 leader_election.cc:288] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [CANDIDATE]: Term 1 election: Requested vote from peers aa2b531682ea49ae9493907554834aee (127.14.51.66:36295), a150ed1ab281453281e7f1604a6e1870 (127.14.51.65:40027)
I20260806 06:41:46.656049 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd5cefe7a284fbd9d69aef8e86909f1" candidate_uuid: "d2650d2eb870482cbb8fd43b564a7489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a150ed1ab281453281e7f1604a6e1870"
I20260806 06:41:46.656050 19634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd5cefe7a284fbd9d69aef8e86909f1" candidate_uuid: "d2650d2eb870482cbb8fd43b564a7489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2b531682ea49ae9493907554834aee"
I20260806 06:41:46.656127 19634 raft_consensus.cc:3037] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:46.656127 19497 raft_consensus.cc:3037] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:46.657029 19497 raft_consensus.cc:2454] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2650d2eb870482cbb8fd43b564a7489 in term 1.
I20260806 06:41:46.657030 19634 raft_consensus.cc:2454] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2650d2eb870482cbb8fd43b564a7489 in term 1.
I20260806 06:41:46.657210 19701 leader_election.cc:302] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2b531682ea49ae9493907554834aee, d2650d2eb870482cbb8fd43b564a7489; no voters:
I20260806 06:41:46.657310 19837 raft_consensus.cc:2781] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:46.657450 19837 raft_consensus.cc:686] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 1 LEADER]: Becoming Leader. State: Replica: d2650d2eb870482cbb8fd43b564a7489, State: Running, Role: LEADER
I20260806 06:41:46.657541 19837 consensus_queue.cc:237] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } }
I20260806 06:41:46.658396 19351 catalog_manager.cc:5495] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2650d2eb870482cbb8fd43b564a7489 (127.14.51.67). New cstate: current_term: 1 leader_uuid: "d2650d2eb870482cbb8fd43b564a7489" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2650d2eb870482cbb8fd43b564a7489" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 41171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:46.694262 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.693312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.14.51.67:41171)
0806 06:41:46.693393 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.693395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.693413 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:41:46.693470 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.693471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.693483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.693533 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.694030 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.694031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.694031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.694035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.694050 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.694054 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.694098 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.694178 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.694191 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:41:46.695990 19497 raft_consensus.cc:1261] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 1 FOLLOWER]: Refusing update from remote peer d2650d2eb870482cbb8fd43b564a7489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:46.696292 19634 raft_consensus.cc:1261] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 1 FOLLOWER]: Refusing update from remote peer d2650d2eb870482cbb8fd43b564a7489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:46.696334 19840 consensus_queue.cc:1035] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a150ed1ab281453281e7f1604a6e1870" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 40027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:46.696588 19840 consensus_queue.cc:1035] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2b531682ea49ae9493907554834aee" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 36295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:46.699484 19844 mvcc.cc:205] Tried to move back new op lower bound from 7315449883424182272 to 7315449883270119424. Current Snapshot: MvccSnapshot[applied={T|T < 7315449883424182272}]
I20260806 06:41:46.699651 19843 mvcc.cc:205] Tried to move back new op lower bound from 7315449883424182272 to 7315449883270119424. Current Snapshot: MvccSnapshot[applied={T|T < 7315449883424182272}]
I20260806 06:41:46.701638 19351 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:46.701740 19351 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:46.702483 19351 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98a749c223b9471983dda55c5384c895: Sending DeleteTablet for 3 replicas of tablet 9fd5cefe7a284fbd9d69aef8e86909f1
I20260806 06:41:46.702826 19477 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:46 UTC) from {username='slave'} at 127.0.0.1:36400
I20260806 06:41:46.702840 19612 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:46 UTC) from {username='slave'} at 127.0.0.1:59624
I20260806 06:41:46.703078 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19414
I20260806 06:41:46.703089 19746 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd5cefe7a284fbd9d69aef8e86909f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:46 UTC) from {username='slave'} at 127.0.0.1:53082
I20260806 06:41:46.703337 19852 tablet_replica.cc:330] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870: stopping tablet replica
I20260806 06:41:46.703444 19852 raft_consensus.cc:2229] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:46.703545 19852 raft_consensus.cc:2258] T 9fd5cefe7a284fbd9d69aef8e86909f1 P a150ed1ab281453281e7f1604a6e1870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:46.704530 19853 tablet_replica.cc:330] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: stopping tablet replica
I20260806 06:41:46.704627 19853 raft_consensus.cc:2229] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:46.704713 19853 raft_consensus.cc:2258] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:46.705753 19853 ts_tablet_manager.cc:1905] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:46.703555 19854 tablet_replica.cc:330] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: stopping tablet replica
I20260806 06:41:46.706519 19854 raft_consensus.cc:2229] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:46.706707 19854 raft_consensus.cc:2258] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:46.707195 19854 ts_tablet_manager.cc:1905] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:46.707623 19853 ts_tablet_manager.cc:1918] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:46.707715 19853 log.cc:1196] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/9fd5cefe7a284fbd9d69aef8e86909f1
I20260806 06:41:46.708006 19853 ts_tablet_manager.cc:1939] T 9fd5cefe7a284fbd9d69aef8e86909f1 P aa2b531682ea49ae9493907554834aee: Deleting consensus metadata
I20260806 06:41:46.708411 19338 catalog_manager.cc:4841] TS aa2b531682ea49ae9493907554834aee (127.14.51.66:36295): tablet 9fd5cefe7a284fbd9d69aef8e86909f1 (table test-table [id=4ce684d92e5245d885452d446d367903]) successfully deleted
I20260806 06:41:46.709477 19854 ts_tablet_manager.cc:1918] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:46.709555 19854 log.cc:1196] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/9fd5cefe7a284fbd9d69aef8e86909f1
I20260806 06:41:46.709816 19854 ts_tablet_manager.cc:1939] T 9fd5cefe7a284fbd9d69aef8e86909f1 P d2650d2eb870482cbb8fd43b564a7489: Deleting consensus metadata
I20260806 06:41:46.710275 19338 catalog_manager.cc:4841] TS d2650d2eb870482cbb8fd43b564a7489 (127.14.51.67:41171): tablet 9fd5cefe7a284fbd9d69aef8e86909f1 (table test-table [id=4ce684d92e5245d885452d446d367903]) successfully deleted
W20260806 06:41:46.710291 19339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:40027 (error 108)
I20260806 06:41:46.710603 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19548
I20260806 06:41:46.710768 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.710655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:40027 (local address 127.0.0.1:36406)
0806 06:41:46.710700 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.710735 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:40027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:41:46.711499 19339 catalog_manager.cc:4584] TS a150ed1ab281453281e7f1604a6e1870 (127.14.51.65:40027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fd5cefe7a284fbd9d69aef8e86909f1: Network error: Client connection negotiation failed: client connection to 127.14.51.65:40027: connect: Connection refused (error 111)
I20260806 06:41:46.717932 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19684
I20260806 06:41:46.723220 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19321
2026-08-06T06:41:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (771 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 06:41:46.742511 14541 mini_oidc.cc:152] Starting JWKS server
I20260806 06:41:46.743793 14541 webserver.cc:460] Webserver started at https://127.0.0.1:43247/ using document root <none> and password file <none>
I20260806 06:41:46.743935 14541 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:41:46.744123 14541 webserver.cc:460] Webserver started at http://127.0.0.1:40487/ using document root <none> and password file <none>
2026-08-06T06:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:46Z Disabled control of system clock
I20260806 06:41:46.751888 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43169
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43169
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/testchainca.cert with env {}
W20260806 06:41:46.830251 19861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:46.830435 19861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:46.830456 19861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:46.831632 19861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:46.831672 19861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:46.831686 19861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:46.831698 19861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:46.831710 19861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:41:46.831753 19861 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:41:46.831782 19861 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43247/jwks.json
I20260806 06:41:46.833295 19861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:46.833542 19861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:46.833760 19861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:46.838569 19867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:46.838578 19866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.838608 19861 server_base.cc:1034] running on GCE node
W20260806 06:41:46.838621 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:46.839061 19861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:46.839442 19861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:46.840590 19861 hybrid_clock.cc:648] HybridClock initialized: now 1785998506840569 us; error 39 us; skew 500 ppm
I20260806 06:41:46.841886 19861 webserver.cc:460] Webserver started at http://127.14.51.126:42211/ using document root <none> and password file <none>
I20260806 06:41:46.842127 19861 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:46.842175 19861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:46.842327 19861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:46.843408 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "db5fd5bf677e45e7bf12eb13b9a93a79"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "0051282abe543dcea2b3449b4f7a7520"
server_key_iv: "8c6b5ce5e407e3530b4b515f9f5a8abb"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.843772 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "db5fd5bf677e45e7bf12eb13b9a93a79"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "0051282abe543dcea2b3449b4f7a7520"
server_key_iv: "8c6b5ce5e407e3530b4b515f9f5a8abb"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.845077 19861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:46.845718 19876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:46.845894 19861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:46.845963 19861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "db5fd5bf677e45e7bf12eb13b9a93a79"
format_stamp: "Formatted at 2026-08-06 06:41:46 on dist-test-slave-1xrh"
server_key: "0051282abe543dcea2b3449b4f7a7520"
server_key_iv: "8c6b5ce5e407e3530b4b515f9f5a8abb"
server_key_version: "encryptionkey@0"
I20260806 06:41:46.846012 19861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:46.861292 19861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:46.861513 19861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:46.861620 19861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:46.865315 19861 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:43169
I20260806 06:41:46.865362 19930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:43169 every 8 connection(s)
I20260806 06:41:46.865676 19861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:46.866631 19931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:46.868733 19931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79: Bootstrap starting.
I20260806 06:41:46.869242 19931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:46.869477 19931 log.cc:824] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:46.870087 19931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79: No bootstrap required, opened a new log
I20260806 06:41:46.871625 19931 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } }
I20260806 06:41:46.871764 19931 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:46.871785 19931 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5fd5bf677e45e7bf12eb13b9a93a79, State: Initialized, Role: FOLLOWER
I20260806 06:41:46.871914 19931 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } }
I20260806 06:41:46.871971 19931 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:46.871991 19931 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:46.872021 19931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:46.872460 19931 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } }
I20260806 06:41:46.872538 19931 leader_election.cc:302] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5fd5bf677e45e7bf12eb13b9a93a79; no voters:
I20260806 06:41:46.872666 19931 leader_election.cc:288] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:46.872720 19934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:46.872879 19934 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [term 1 LEADER]: Becoming Leader. State: Replica: db5fd5bf677e45e7bf12eb13b9a93a79, State: Running, Role: LEADER
I20260806 06:41:46.872920 19931 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:46.872977 19934 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } }
I20260806 06:41:46.873471 19936 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5fd5bf677e45e7bf12eb13b9a93a79. Latest consensus state: current_term: 1 leader_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } } }
I20260806 06:41:46.873564 19936 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:46.873735 19935 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5fd5bf677e45e7bf12eb13b9a93a79" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43169 } } }
I20260806 06:41:46.873795 19935 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:46.873903 19943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:46.874501 19943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:46.875870 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19861
I20260806 06:41:46.875972 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:46.876165 14541 external_mini_cluster.cc:1428] Setting key 2a7b0200947e17e488996eb165505f0a
I20260806 06:41:46.876222 19943 catalog_manager.cc:1347] Generated new cluster ID: ccf855c75ce548eeb80144ce531bddde
I20260806 06:41:46.876267 19943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:46.881178 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.876608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.14.51.126:43169)
0806 06:41:46.876999 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.877006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.877028 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:41:46.877112 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.877117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.877171 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.877302 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:46.877551 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.878435 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.879303 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.879715 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.879860 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:46.880488 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:46.880491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:46.880491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:46.880659 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:46.880690 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:46.880699 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:46.880769 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:46.880860 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:41:46.881022 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":73,"threads_started":1}
I20260806 06:41:46.881628 19943 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:46.882067 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:41:46.882247 19943 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:46.885440 19943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5fd5bf677e45e7bf12eb13b9a93a79: Generated new TSK 0
I20260806 06:41:46.885610 19943 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:41:46.911834 19963 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 06:41:46.911969 19963 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 06:41:46.912549 19962 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:46.895667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43169 (local address 127.0.0.1:41774)
0806 06:41:46.895890 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:46.895899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:46.895952 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:46.896110 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:46.896111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:46.896112 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:41:46.896159 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:46.896161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.897095 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.897095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:46.897672 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:46.897675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.898171 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.898172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:46.898223 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:46.898241 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:41:46.912090 (+ 13849us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:41:46.912174 (+ 84us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.51.126:43169: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260806 06:41:46.912927 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:46.895747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.14.51.126:43169)
0806 06:41:46.895785 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:46.895787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:46.895959 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:41:46.896015 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:46.896015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:46.896026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:46.896099 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:41:46.896219 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.896988 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.897755 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:46.898063 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:46.898146 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:46.898357 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:41:46.911953 (+ 13596us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:41:46.912761 (+ 808us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:41:46.912981 19955 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.51.126:43169: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 06:41:46.913057 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19861
2026-08-06T06:41:46Z chronyd exiting
I20260806 06:41:46.950573 14541 test_util.cc:175] -----------------------------------------------
I20260806 06:41:46.950671 14541 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998482572191-14541-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 06:41:46.951516 14541 mini_oidc.cc:152] Starting JWKS server
I20260806 06:41:46.952191 14541 webserver.cc:460] Webserver started at https://127.0.0.1:37689/ using document root <none> and password file <none>
I20260806 06:41:46.952255 14541 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:41:46.952399 14541 webserver.cc:460] Webserver started at http://127.0.0.1:38935/ using document root <none> and password file <none>
2026-08-06T06:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:46Z Disabled control of system clock
I20260806 06:41:46.960182 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:45015
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:45015
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37689/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/testchainca.cert with env {}
W20260806 06:41:47.039242 19970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:47.039418 19970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:47.039438 19970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:47.040511 19970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:47.040552 19970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:47.040565 19970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:47.040577 19970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:47.040589 19970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:41:47.040601 19970 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:41:47.040612 19970 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37689/jwks.json
I20260806 06:41:47.042047 19970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:45015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37689/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:47.042250 19970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:47.042395 19970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:47.047013 19975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:47.047024 19976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:47.047024 19978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:47.047255 19970 server_base.cc:1034] running on GCE node
I20260806 06:41:47.047480 19970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:47.047762 19970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:47.048905 19970 hybrid_clock.cc:648] HybridClock initialized: now 1785998507048887 us; error 29 us; skew 500 ppm
I20260806 06:41:47.050068 19970 webserver.cc:460] Webserver started at http://127.14.51.126:45929/ using document root <none> and password file <none>
I20260806 06:41:47.050302 19970 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:47.050370 19970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:47.050520 19970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:47.051600 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "9ddde5dd65dc45bf81e3228067441baa"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "840b01155175489a08e3547bfa286f66"
server_key_iv: "d8c436c4443068078c69713571d69ab4"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.051952 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "9ddde5dd65dc45bf81e3228067441baa"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "840b01155175489a08e3547bfa286f66"
server_key_iv: "d8c436c4443068078c69713571d69ab4"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.053251 19970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:47.053970 19984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:47.054147 19970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:47.054240 19970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "9ddde5dd65dc45bf81e3228067441baa"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "840b01155175489a08e3547bfa286f66"
server_key_iv: "d8c436c4443068078c69713571d69ab4"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.054324 19970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:47.080329 19970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:47.080660 19970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:47.080821 19970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:47.084784 19970 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:45015
I20260806 06:41:47.084823 20037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:45015 every 8 connection(s)
I20260806 06:41:47.085170 19970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:47.085798 20038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:47.088550 20038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa: Bootstrap starting.
I20260806 06:41:47.089116 20038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:47.089404 20038 log.cc:824] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:47.090066 20038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa: No bootstrap required, opened a new log
I20260806 06:41:47.091765 20038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } }
I20260806 06:41:47.091897 20038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:47.091922 20038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ddde5dd65dc45bf81e3228067441baa, State: Initialized, Role: FOLLOWER
I20260806 06:41:47.092097 20038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } }
I20260806 06:41:47.092178 20038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:47.092218 20038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:47.092269 20038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:47.092769 20038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } }
I20260806 06:41:47.092880 20038 leader_election.cc:302] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ddde5dd65dc45bf81e3228067441baa; no voters:
I20260806 06:41:47.093060 20038 leader_election.cc:288] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:47.093107 20041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:47.093276 20041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [term 1 LEADER]: Becoming Leader. State: Replica: 9ddde5dd65dc45bf81e3228067441baa, State: Running, Role: LEADER
I20260806 06:41:47.093317 20038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:47.093366 20041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } }
I20260806 06:41:47.093762 20042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ddde5dd65dc45bf81e3228067441baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } } }
I20260806 06:41:47.093860 20042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.093945 20043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ddde5dd65dc45bf81e3228067441baa. Latest consensus state: current_term: 1 leader_uuid: "9ddde5dd65dc45bf81e3228067441baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ddde5dd65dc45bf81e3228067441baa" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 45015 } } }
I20260806 06:41:47.094033 20043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.094322 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 19970
I20260806 06:41:47.094458 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:47.094473 20050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:47.094673 14541 external_mini_cluster.cc:1428] Setting key ae212b3f7b5f62b022c97e51d002454c
I20260806 06:41:47.094950 20050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:47.096633 20050 catalog_manager.cc:1347] Generated new cluster ID: 39fd9b365afa46128576a03c3dcccd97
I20260806 06:41:47.096686 20050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:47.100893 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.095309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.14.51.126:45015)
0806 06:41:47.095715 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.095723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.095758 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:41:47.095862 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.095868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.095919 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.096069 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:47.096276 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.097638 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.098762 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.099377 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.099492 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.100217 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:47.100220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:47.100221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:47.100420 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:47.100459 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:47.100473 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:47.100533 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:47.100634 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:41:47.100700 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":76,"threads_started":1}
I20260806 06:41:47.101752 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:41:47.116902 20050 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:47.117568 20050 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:47.121227 20050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ddde5dd65dc45bf81e3228067441baa: Generated new TSK 0
I20260806 06:41:47.121387 20050 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:41:47.139494 20068 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 06:41:47.139582 20068 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:41:47.139887 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.125767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.14.51.126:45015)
0806 06:41:47.125838 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.125840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.125893 (+ 53us) server_negotiation.cc:407] Connection header received
0806 06:41:47.125964 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.125965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.125977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.126041 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:41:47.126164 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.126963 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.127733 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.128057 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.128130 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.128287 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:41:47.139694 (+ 11407us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:41:47.139823 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 06:41:47.139940 20060 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 06:41:47.139969 20067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:47.125688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:45015 (local address 127.0.0.1:36844)
0806 06:41:47.125833 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:47.125844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:47.125906 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:47.126045 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:47.126046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:47.126047 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:41:47.126098 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.126099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.127033 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.127034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.127655 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.127657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.128142 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.128143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.128186 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.128190 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:41:47.139801 (+ 11611us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:41:47.139875 (+ 74us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.51.126:45015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 06:41:47.140231 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 19970
2026-08-06T06:41:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 06:41:47.178442 14541 mini_oidc.cc:152] Starting JWKS server
I20260806 06:41:47.179167 14541 webserver.cc:460] Webserver started at https://127.0.0.1:41577/ using document root <none> and password file <none>
I20260806 06:41:47.179227 14541 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:41:47.179448 14541 webserver.cc:460] Webserver started at http://127.0.0.1:38925/ using document root <none> and password file <none>
2026-08-06T06:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:47Z Disabled control of system clock
I20260806 06:41:47.186942 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33401
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33401
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41577/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:41:47.265079 20074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:47.265306 20074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:47.265344 20074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:47.266427 20074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:47.266503 20074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:47.266544 20074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:47.266572 20074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:47.266597 20074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:41:47.266626 20074 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:41:47.266649 20074 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41577/jwks.json
I20260806 06:41:47.268003 20074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41577/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:47.268256 20074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:47.268532 20074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:47.273219 20079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:47.273277 20080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:47.273401 20074 server_base.cc:1034] running on GCE node
W20260806 06:41:47.273521 20082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:47.273787 20074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:47.274067 20074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:47.275183 20074 hybrid_clock.cc:648] HybridClock initialized: now 1785998507275170 us; error 28 us; skew 500 ppm
I20260806 06:41:47.276425 20074 webserver.cc:460] Webserver started at http://127.14.51.126:46431/ using document root <none> and password file <none>
I20260806 06:41:47.276681 20074 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:47.276751 20074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:47.276927 20074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:47.277925 20074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "f6ec50e90b0743e79bea3acde6ee2b3b"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "cbf1ed8bc28afe467b7b56d10797f9c2"
server_key_iv: "bcb066f053f2a8f95665850a5aff877d"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.278276 20074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "f6ec50e90b0743e79bea3acde6ee2b3b"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "cbf1ed8bc28afe467b7b56d10797f9c2"
server_key_iv: "bcb066f053f2a8f95665850a5aff877d"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.279660 20074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:47.280413 20088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:47.280618 20074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:47.280699 20074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "f6ec50e90b0743e79bea3acde6ee2b3b"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "cbf1ed8bc28afe467b7b56d10797f9c2"
server_key_iv: "bcb066f053f2a8f95665850a5aff877d"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.280748 20074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:47.308867 20074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:47.309093 20074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:47.309191 20074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:47.314045 20074 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:33401
I20260806 06:41:47.314139 20141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:33401 every 8 connection(s)
I20260806 06:41:47.314520 20074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:47.315369 20142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:47.318409 20142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b: Bootstrap starting.
I20260806 06:41:47.318959 20142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:47.319312 20142 log.cc:824] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:47.319942 20142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b: No bootstrap required, opened a new log
I20260806 06:41:47.321178 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20074
I20260806 06:41:47.321285 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:47.321475 14541 external_mini_cluster.cc:1428] Setting key e1dbc7a1e8a0d46c51517cfb2dbdd3e8
I20260806 06:41:47.322005 20142 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } }
I20260806 06:41:47.322206 20142 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:47.322260 20142 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ec50e90b0743e79bea3acde6ee2b3b, State: Initialized, Role: FOLLOWER
I20260806 06:41:47.322389 20142 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } }
I20260806 06:41:47.322475 20142 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:47.322496 20142 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:47.322566 20142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:47.323067 20142 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } }
I20260806 06:41:47.323220 20142 leader_election.cc:302] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6ec50e90b0743e79bea3acde6ee2b3b; no voters:
I20260806 06:41:47.323393 20142 leader_election.cc:288] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:47.323614 20147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:47.323693 20142 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:47.323757 20147 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [term 1 LEADER]: Becoming Leader. State: Replica: f6ec50e90b0743e79bea3acde6ee2b3b, State: Running, Role: LEADER
I20260806 06:41:47.323853 20147 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } }
I20260806 06:41:47.324209 20147 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ec50e90b0743e79bea3acde6ee2b3b. Latest consensus state: current_term: 1 leader_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } } }
I20260806 06:41:47.324303 20147 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.324626 20148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec50e90b0743e79bea3acde6ee2b3b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33401 } } }
I20260806 06:41:47.324710 20148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.325021 20156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:47.325512 20156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:47.327333 20156 catalog_manager.cc:1347] Generated new cluster ID: 633f452040c0430d965a362a39dd0a94
I20260806 06:41:47.327394 20156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:47.327837 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.322096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.14.51.126:33401)
0806 06:41:47.322341 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.322355 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.322377 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:41:47.322514 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.322521 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.322586 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.322722 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:47.322940 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.324201 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.325503 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.326022 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.326147 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.327041 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:47.327045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:47.327046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:47.327265 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:47.327307 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:47.327321 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:47.327384 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:47.327579 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 06:41:47.327649 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260806 06:41:47.332405 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:41:47.337402 20156 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:47.338083 20156 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:47.341214 20156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6ec50e90b0743e79bea3acde6ee2b3b: Generated new TSK 0
I20260806 06:41:47.341374 20156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:47.353132 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.349996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42874 (local address 127.14.51.126:33401)
0806 06:41:47.350039 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.350041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.350121 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:41:47.350218 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.350219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.350230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.350314 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:41:47.350453 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.350934 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.351732 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.351944 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.352025 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.352205 (+ 180us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:41:47.352936 (+ 731us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41577/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:41:47.353075 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42874: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41577/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:41:47.353276 20146 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42874: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41577/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 06:41:47.353299 20173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:47.349906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33401 (local address 127.0.0.1:42874)
0806 06:41:47.350044 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:47.350055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:47.350139 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:47.350312 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:47.350313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:47.350315 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:41:47.350387 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.350388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.351013 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.351014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.351611 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.351613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.352029 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.352031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.352085 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.352089 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:41:47.353027 (+ 938us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41577/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:41:47.353192 (+ 165us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.51.126:33401: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41577/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":44,"thread_start_us":90,"threads_started":1}
I20260806 06:41:47.353507 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20074
E20260806 06:41:47.365304 20174 webserver.cc:533] Webserver: sslize failed:
E20260806 06:41:47.365362 20174 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T06:41:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T06:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:47Z Disabled control of system clock
I20260806 06:41:47.399792 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40175
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:40175
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/testchainca.cert with env {}
W20260806 06:41:47.478425 20178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:47.478615 20178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:47.478636 20178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:47.479737 20178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:47.479773 20178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:47.479785 20178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:47.479799 20178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:47.479810 20178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:41:47.479822 20178 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:41:47.479840 20178 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 06:41:47.481182 20178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:40175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:47.481346 20178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:47.481479 20178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:47.486123 20184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:47.486173 20178 server_base.cc:1034] running on GCE node
W20260806 06:41:47.486135 20186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:47.486131 20183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:47.486567 20178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:47.486850 20178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:47.488022 20178 hybrid_clock.cc:648] HybridClock initialized: now 1785998507488003 us; error 36 us; skew 500 ppm
I20260806 06:41:47.489287 20178 webserver.cc:460] Webserver started at http://127.14.51.126:44369/ using document root <none> and password file <none>
I20260806 06:41:47.489516 20178 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:47.489583 20178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:47.489711 20178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:47.490638 20178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "8a40602cf0714928823145a81d6530d9"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "8eed5e14d4c6224d1e59f48873bc4319"
server_key_iv: "8055280c162a36d2f0c651a792a3212b"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.490976 20178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "8a40602cf0714928823145a81d6530d9"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "8eed5e14d4c6224d1e59f48873bc4319"
server_key_iv: "8055280c162a36d2f0c651a792a3212b"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.492331 20178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:47.493016 20192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:47.493178 20178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:47.493273 20178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "8a40602cf0714928823145a81d6530d9"
format_stamp: "Formatted at 2026-08-06 06:41:47 on dist-test-slave-1xrh"
server_key: "8eed5e14d4c6224d1e59f48873bc4319"
server_key_iv: "8055280c162a36d2f0c651a792a3212b"
server_key_version: "encryptionkey@0"
I20260806 06:41:47.493350 20178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:47.519338 20178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:47.519608 20178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:47.519774 20178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:47.523244 20178 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:40175
I20260806 06:41:47.523290 20245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:40175 every 8 connection(s)
I20260806 06:41:47.523605 20178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:47.524514 20246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:47.526556 20246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9: Bootstrap starting.
I20260806 06:41:47.527047 20246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:47.527356 20246 log.cc:824] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:47.527952 20246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9: No bootstrap required, opened a new log
I20260806 06:41:47.529419 20246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } }
I20260806 06:41:47.529553 20246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:47.529577 20246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a40602cf0714928823145a81d6530d9, State: Initialized, Role: FOLLOWER
I20260806 06:41:47.529707 20246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } }
I20260806 06:41:47.529790 20246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:47.529814 20246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:47.529840 20246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:47.530287 20246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } }
I20260806 06:41:47.530367 20246 leader_election.cc:302] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a40602cf0714928823145a81d6530d9; no voters:
I20260806 06:41:47.530479 20246 leader_election.cc:288] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:47.530525 20249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:47.530663 20249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [term 1 LEADER]: Becoming Leader. State: Replica: 8a40602cf0714928823145a81d6530d9, State: Running, Role: LEADER
I20260806 06:41:47.530766 20246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:47.530753 20249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } }
I20260806 06:41:47.531194 20250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a40602cf0714928823145a81d6530d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } } }
I20260806 06:41:47.531280 20250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.531370 20251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a40602cf0714928823145a81d6530d9. Latest consensus state: current_term: 1 leader_uuid: "8a40602cf0714928823145a81d6530d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40602cf0714928823145a81d6530d9" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40175 } } }
I20260806 06:41:47.531519 20251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:47.531674 20255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:47.532227 20255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:47.533738 20255 catalog_manager.cc:1347] Generated new cluster ID: 81ee06eaee7d4f52aa4a071851ce1ef6
I20260806 06:41:47.533794 20255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:47.533872 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20178
I20260806 06:41:47.533972 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:47.534154 14541 external_mini_cluster.cc:1428] Setting key a4c7743efeec08673473dea259966933
I20260806 06:41:47.539392 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.534558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.14.51.126:40175)
0806 06:41:47.535275 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.535282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.535305 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:41:47.535397 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.535402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.535450 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.535582 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:47.535789 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.536675 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.537450 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.537872 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.537990 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.538660 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:47.538664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:47.538664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:41:47.538841 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:47.538869 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:41:47.538878 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:47.538970 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:47.539049 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:41:47.539256 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":76,"threads_started":1}
I20260806 06:41:47.540297 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:41:47.546036 20255 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:47.546697 20255 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:47.554553 20255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a40602cf0714928823145a81d6530d9: Generated new TSK 0
I20260806 06:41:47.554721 20255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:48.096462 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:47.564010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.14.51.126:40175)
0806 06:41:47.564051 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:47.564053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:47.564155 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:41:47.564209 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:47.564210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:47.564222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:47.564307 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:41:47.564442 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.565193 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.565906 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.566218 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.566284 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.566436 (+ 152us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:41:48.096195 (+529759us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:41:48.096363 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:41:48.096527 20269 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 06:41:48.096635 20275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:47.563932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40175 (local address 127.0.0.1:42394)
0806 06:41:47.564079 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:47.564096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:47.564147 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:47.564311 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:47.564311 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:47.564313 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:41:47.564375 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.564377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.565264 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.565264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.565828 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:47.565830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:47.566295 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:47.566296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:47.566338 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:47.566342 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:41:48.096334 (+529992us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:41:48.096492 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.51.126:40175: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260806 06:41:48.097113 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20178
2026-08-06T06:41:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (723 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:48 dist-test-slave-1xrh krb5kdc[20287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:48 dist-test-slave-1xrh krb5kdc[20287](info): set up 2 sockets
Aug 06 06:41:48 dist-test-slave-1xrh krb5kdc[20287](info): commencing operation
krb5kdc: starting...
W20260806 06:41:49.334398 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:41:49.447820 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40589
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:49.525481 20301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:49.525738 20301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:49.525775 20301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:49.526949 20301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:49.527019 20301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:49.527042 20301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:49.527065 20301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:49.527098 20301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:49.528597 20301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:49.528877 20301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:49.529100 20301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:41:49.534031 20301 server_base.cc:1034] running on GCE node
W20260806 06:41:49.534039 20307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:49.533987 20306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:49.534039 20309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:49.535426 20301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:49.535712 20301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:49.536877 20301 hybrid_clock.cc:648] HybridClock initialized: now 1785998509536862 us; error 46 us; skew 500 ppm
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:49.539402 20301 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:41:49.539749 20301 webserver.cc:460] Webserver started at http://127.14.51.126:46331/ using document root <none> and password file <none>
I20260806 06:41:49.539945 20301 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:49.539985 20301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:49.540068 20301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:49.540973 20301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "79dee1d06c864c2b90867fb8e3ec6e53"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "e95de0c2e6bee4c103c89ff32015ccfc"
server_key_iv: "9b90a7247e1a53808fe3b35a262d78f5"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.541287 20301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "79dee1d06c864c2b90867fb8e3ec6e53"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "e95de0c2e6bee4c103c89ff32015ccfc"
server_key_iv: "9b90a7247e1a53808fe3b35a262d78f5"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.542508 20301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:49.543193 20316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.543372 20301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:49.543457 20301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "79dee1d06c864c2b90867fb8e3ec6e53"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "e95de0c2e6bee4c103c89ff32015ccfc"
server_key_iv: "9b90a7247e1a53808fe3b35a262d78f5"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.543511 20301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:49.564584 20301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:49.566172 20301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:49.566274 20301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:49.569883 20301 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:40589
I20260806 06:41:49.569931 20368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:40589 every 8 connection(s)
I20260806 06:41:49.570240 20301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:49.571030 20369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:49.572032 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20301
I20260806 06:41:49.572121 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:49.572338 14541 external_mini_cluster.cc:1428] Setting key c377cae8cc94ceeb29e2b5d90a3fe6d6
I20260806 06:41:49.573518 20369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53: Bootstrap starting.
I20260806 06:41:49.574097 20369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:49.574384 20369 log.cc:824] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:49.575110 20369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53: No bootstrap required, opened a new log
I20260806 06:41:49.576799 20369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } }
I20260806 06:41:49.576946 20369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:49.576994 20369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79dee1d06c864c2b90867fb8e3ec6e53, State: Initialized, Role: FOLLOWER
I20260806 06:41:49.577147 20369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } }
I20260806 06:41:49.577234 20369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:49.577271 20369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:49.577325 20369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:49.577836 20369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } }
I20260806 06:41:49.577919 20369 leader_election.cc:302] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79dee1d06c864c2b90867fb8e3ec6e53; no voters:
I20260806 06:41:49.578148 20369 leader_election.cc:288] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:49.578167 20374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:49.578289 20374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [term 1 LEADER]: Becoming Leader. State: Replica: 79dee1d06c864c2b90867fb8e3ec6e53, State: Running, Role: LEADER
I20260806 06:41:49.578365 20374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } }
I20260806 06:41:49.578387 20369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:49.578665 20374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } } }
I20260806 06:41:49.578790 20374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [sys.catalog]: This master's current role is: LEADER
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:49.578990 20374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79dee1d06c864c2b90867fb8e3ec6e53. Latest consensus state: current_term: 1 leader_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dee1d06c864c2b90867fb8e3ec6e53" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 40589 } } }
I20260806 06:41:49.579047 20374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:49.579399 20383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:49.579895 20383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:49.581676 20383 catalog_manager.cc:1347] Generated new cluster ID: 669dd11dfb8841fb89351e7d0fff12cf
I20260806 06:41:49.581736 20383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:49.583221 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:49.572850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.14.51.126:40589)
0806 06:41:49.573048 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:49.573055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:49.573079 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:41:49.573485 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:49.573490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:49.573537 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:49.573714 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:49.574444 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.575631 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.576381 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.576779 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.576893 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:49.579617 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:49.579621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:49.579621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:49.579632 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:49.581558 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.581985 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.581988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.581988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.582089 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.582304 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.582305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.582305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.582567 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:49.582710 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:49.582889 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 06:41:49.583012 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260806 06:41:49.589886 20383 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:49.590549 20383 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:49.597641 20383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79dee1d06c864c2b90867fb8e3ec6e53: Generated new TSK 0
I20260806 06:41:49.597793 20383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:41:49.628019 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:49.706717 20396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:49.706938 20396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:49.706979 20396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:49.708163 20396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:49.708225 20396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:49.708268 20396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:49.709654 20396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:49.709903 20396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:49.710114 20396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:49.714828 20404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:49.714893 20402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:49.714985 20396 server_base.cc:1034] running on GCE node
W20260806 06:41:49.714883 20401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:49.715319 20396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:49.715528 20396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:49.716686 20396 hybrid_clock.cc:648] HybridClock initialized: now 1785998509716670 us; error 44 us; skew 500 ppm
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:49.719265 20396 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:41:49.719698 20396 webserver.cc:460] Webserver started at http://127.14.51.65:36267/ using document root <none> and password file <none>
I20260806 06:41:49.719920 20396 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:49.719985 20396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:49.720115 20396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:49.721062 20396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "7ba7fd26c4584bb78ea677c7ef8b9947"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "1e631e2a19582f988f828632993958ea"
server_key_iv: "619144867bc11bf22a7f7d716aa65f79"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.721407 20396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba7fd26c4584bb78ea677c7ef8b9947"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "1e631e2a19582f988f828632993958ea"
server_key_iv: "619144867bc11bf22a7f7d716aa65f79"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.722633 20396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:49.723359 20411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.723523 20396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:49.723635 20396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "7ba7fd26c4584bb78ea677c7ef8b9947"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "1e631e2a19582f988f828632993958ea"
server_key_iv: "619144867bc11bf22a7f7d716aa65f79"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.723721 20396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:49.737452 20396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:49.739672 20396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:49.739821 20396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:49.740036 20396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:49.740315 20396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:49.740365 20396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.740396 20396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:49.740425 20396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.746222 20396 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:39649
I20260806 06:41:49.746292 20528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:39649 every 8 connection(s)
I20260806 06:41:49.746590 20396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:49.752619 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20396
I20260806 06:41:49.752754 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:49.752941 14541 external_mini_cluster.cc:1428] Setting key 34493400337205b2a5a8ac18b31372c0
I20260806 06:41:49.755358 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:49.747331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:42905 (local address 127.14.51.126:40589)
0806 06:41:49.747419 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:49.747421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:49.747552 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:41:49.748414 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:49.748415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:49.748424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:49.748473 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:49.748994 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.749475 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.750390 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.750578 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.750626 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:49.752436 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:49.752437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:49.752437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:49.752444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:49.753986 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.754397 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.754398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.754399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.754460 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.754704 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.754705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.754705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.754829 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:49.754909 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:49.755185 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 06:41:49.755272 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:41:49.755509 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:49.747283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40589 (local address 127.14.51.65:42905)
0806 06:41:49.747437 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:49.747471 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:49.748329 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:49.748522 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:49.748526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:49.748753 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:49.748927 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:49.748931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.749556 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.749556 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:49.750307 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:49.750309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.750647 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.750648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:49.750744 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:49.751104 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:49.751111 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:49.752358 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:49.754082 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:49.754088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:49.754090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:49.754304 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:49.754525 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:49.754525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:49.754526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:49.754635 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:49.754894 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:49.754900 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:49.755031 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:41:49.755128 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:41:49.755288 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260806 06:41:49.756141 20529 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40589
I20260806 06:41:49.756238 20529 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:49.756417 20529 heartbeater.cc:511] Master 127.14.51.126:40589 requested a full tablet report, sending...
I20260806 06:41:49.756935 20333 ts_manager.cc:194] Registered new tserver with Master: 7ba7fd26c4584bb78ea677c7ef8b9947 (127.14.51.65:39649)
I20260806 06:41:49.757884 20333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:42905
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:41:49.777457 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:49.856951 20535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:49.857142 20535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:49.857164 20535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:49.858244 20535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:49.858280 20535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:49.858305 20535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:49.859694 20535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:49.859866 20535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:49.860051 20535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:49.864928 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:49.865051 20535 server_base.cc:1034] running on GCE node
W20260806 06:41:49.864970 20541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:49.864985 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:49.865339 20535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:49.865526 20535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:49.866705 20535 hybrid_clock.cc:648] HybridClock initialized: now 1785998509866689 us; error 30 us; skew 500 ppm
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:49.869271 20535 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:41:49.869688 20535 webserver.cc:460] Webserver started at http://127.14.51.66:34047/ using document root <none> and password file <none>
I20260806 06:41:49.869922 20535 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:49.869990 20535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:49.870138 20535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:49.871219 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "9e370c0a16cb41948500cd5959b3ae8e"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "15d371c70b7597d81c54aed96acba333"
server_key_iv: "6625c94f6696f7ff64d8f79071fb31be"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.871582 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "9e370c0a16cb41948500cd5959b3ae8e"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "15d371c70b7597d81c54aed96acba333"
server_key_iv: "6625c94f6696f7ff64d8f79071fb31be"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.872830 20535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:49.873571 20551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.873745 20535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:49.873842 20535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "9e370c0a16cb41948500cd5959b3ae8e"
format_stamp: "Formatted at 2026-08-06 06:41:49 on dist-test-slave-1xrh"
server_key: "15d371c70b7597d81c54aed96acba333"
server_key_iv: "6625c94f6696f7ff64d8f79071fb31be"
server_key_version: "encryptionkey@0"
I20260806 06:41:49.873927 20535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:49.895488 20535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:49.897223 20535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:49.897323 20535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:49.897568 20535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:49.897904 20535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:49.897953 20535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.897972 20535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:49.898020 20535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:49.903535 20535 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:44509
I20260806 06:41:49.903595 20671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:44509 every 8 connection(s)
I20260806 06:41:49.903898 20535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:49 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998509, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:49.912096 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20535
I20260806 06:41:49.912201 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:49.912390 14541 external_mini_cluster.cc:1428] Setting key 3ff95bed215fbdf2367e84f340e18919
I20260806 06:41:49.912940 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:49.904611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:46111 (local address 127.14.51.126:40589)
0806 06:41:49.904653 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:49.904655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:49.904870 (+ 215us) server_negotiation.cc:407] Connection header received
0806 06:41:49.905736 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:49.905737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:49.905747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:49.905799 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:49.906343 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.906829 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.907746 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.907935 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.907987 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:49.910115 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:49.910117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:49.910117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:49.910124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:49.911638 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.911997 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.911998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.911999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.912061 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:49.912318 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:49.912319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:49.912320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:49.912450 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:49.912517 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:49.912779 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 06:41:49.912851 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:49.913101 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:49.904579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40589 (local address 127.14.51.66:46111)
0806 06:41:49.904745 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:49.904786 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:49.905650 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:49.905857 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:49.905860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:49.906103 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:49.906277 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:49.906280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.906905 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.906906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:49.907678 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:49.907680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:49.908010 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:49.908011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:49.908103 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:49.908680 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:49.908688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:49.910018 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:49.911703 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:49.911708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:49.911709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:49.911925 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:49.912126 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:49.912126 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:49.912127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:49.912263 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:49.912509 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:49.912514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:49.912670 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 06:41:49.912757 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:41:49.912862 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260806 06:41:49.914741 20672 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40589
I20260806 06:41:49.914847 20672 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:49.915030 20672 heartbeater.cc:511] Master 127.14.51.126:40589 requested a full tablet report, sending...
I20260806 06:41:49.915477 20333 ts_manager.cc:194] Registered new tserver with Master: 9e370c0a16cb41948500cd5959b3ae8e (127.14.51.66:44509)
I20260806 06:41:49.915973 20333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:46111
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:41:49.937326 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:50.018528 20677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:50.018725 20677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:50.018748 20677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:50.019902 20677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:50.019943 20677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:50.019969 20677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:50.021494 20677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:50.021742 20677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:50.022019 20677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:50.026944 20682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:50.026952 20685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:50.026973 20683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:50.027253 20677 server_base.cc:1034] running on GCE node
I20260806 06:41:50.027496 20677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:50.027731 20677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:50.028913 20677 hybrid_clock.cc:648] HybridClock initialized: now 1785998510028894 us; error 26 us; skew 500 ppm
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.031456 20677 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:41:50.031867 20677 webserver.cc:460] Webserver started at http://127.14.51.67:34739/ using document root <none> and password file <none>
I20260806 06:41:50.032087 20677 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:50.032132 20677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:50.032274 20677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:50.033268 20677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "b40ec31b0b794daf87c3b817554892e2"
format_stamp: "Formatted at 2026-08-06 06:41:50 on dist-test-slave-1xrh"
server_key: "85e43be3be50dc0b6454d3b9e780d4c8"
server_key_iv: "87d633c4279d1215301946698ae6782e"
server_key_version: "encryptionkey@0"
I20260806 06:41:50.033617 20677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "b40ec31b0b794daf87c3b817554892e2"
format_stamp: "Formatted at 2026-08-06 06:41:50 on dist-test-slave-1xrh"
server_key: "85e43be3be50dc0b6454d3b9e780d4c8"
server_key_iv: "87d633c4279d1215301946698ae6782e"
server_key_version: "encryptionkey@0"
I20260806 06:41:50.034899 20677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:50.035657 20692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:50.035826 20677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:50.035908 20677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "b40ec31b0b794daf87c3b817554892e2"
format_stamp: "Formatted at 2026-08-06 06:41:50 on dist-test-slave-1xrh"
server_key: "85e43be3be50dc0b6454d3b9e780d4c8"
server_key_iv: "87d633c4279d1215301946698ae6782e"
server_key_version: "encryptionkey@0"
I20260806 06:41:50.035959 20677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:50.046998 20677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:50.048840 20677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:50.048942 20677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:50.049099 20677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:50.049340 20677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:50.049371 20677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:50.049391 20677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:50.049402 20677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:50.054971 20677 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:39509
I20260806 06:41:50.055049 20811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:39509 every 8 connection(s)
I20260806 06:41:50.055439 20677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:41:50.061591 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20677
I20260806 06:41:50.061681 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:50.061851 14541 external_mini_cluster.cc:1428] Setting key afce11c9947af6214e7ef993cdaafee2
I20260806 06:41:50.064810 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:50.056158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:59861 (local address 127.14.51.126:40589)
0806 06:41:50.056225 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.056227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.056435 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:41:50.057367 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.057369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.057381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.057447 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.058029 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.058544 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.059597 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.059825 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.059909 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:50.062172 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:50.062174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:50.062175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:50.062182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:50.063705 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.064066 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.064066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.064067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.064114 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.064316 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.064316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.064317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.064419 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:50.064467 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:50.064690 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:41:50.064747 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:41:50.065022 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:50.056140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:40589 (local address 127.14.51.67:59861)
0806 06:41:50.056314 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:50.056347 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:50.057256 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:50.057512 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:50.057520 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:50.057756 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:50.057936 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:50.057940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.058638 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.058639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:50.059511 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:50.059513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.059925 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.059926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:50.060039 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:50.060541 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:50.060549 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:50.062085 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:50.063774 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:50.063779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:50.063780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:50.064001 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:50.064173 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:50.064173 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:50.064173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:50.064264 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:50.064475 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:50.064479 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:50.064585 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 06:41:50.064666 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:41:50.064778 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260806 06:41:50.065491 20812 heartbeater.cc:347] Connected to a master server at 127.14.51.126:40589
I20260806 06:41:50.065593 20812 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:50.065727 20812 heartbeater.cc:511] Master 127.14.51.126:40589 requested a full tablet report, sending...
I20260806 06:41:50.066083 20333 ts_manager.cc:194] Registered new tserver with Master: b40ec31b0b794daf87c3b817554892e2 (127.14.51.67:39509)
I20260806 06:41:50.066442 20333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:59861
I20260806 06:41:50.073539 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/test-admin.keytab.
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.089555 14541 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 06:41:50.091110 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:50.090327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.14.51.65:39649)
0806 06:41:50.090486 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.090491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.090523 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:41:50.090554 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.090556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.090569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.090636 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.091008 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
W20260806 06:41:50.091315 14541 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.092203 14541 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.093073 14541 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:50.100378 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:50.093531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.14.51.65:39649)
0806 06:41:50.093583 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.093585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.093601 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:41:50.093621 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.093622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.093630 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.093680 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.094036 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.094743 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.095493 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.095815 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.095871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:50.097941 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:50.097949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:50.097949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:50.097955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:50.099322 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.099629 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.099632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.099632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.099680 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.099834 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.099835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.099835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.099988 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:50.100087 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:50.100167 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:41:50.100227 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 06:41:50.102068 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:50.101536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.14.51.65:39649)
0806 06:41:50.101593 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.101595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.101603 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:50.101661 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.101662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.101671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.101718 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.102023 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41948: BlockingRecv error: recv got EOF from 127.0.0.1:41948 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:41:50.102206 14541 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.103111 14541 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.104027 14541 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:50.110366 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:50.104500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.14.51.65:39649)
0806 06:41:50.104557 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.104559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.104594 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:41:50.104609 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.104610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.104616 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.104661 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.104996 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.105447 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.106054 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.106233 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.106281 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:50.108164 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:50.108165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:50.108166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:50.108172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:50.109273 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.109649 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.109650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.109650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.109706 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.109915 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.109916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.109916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.110007 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:50.110102 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:50.110207 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:41:50.110279 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 06:41:50.111928 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:50.111409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.14.51.65:39649)
0806 06:41:50.111440 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.111442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.111511 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:41:50.111584 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.111584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.111592 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.111636 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.111886 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41954: BlockingRecv error: recv got EOF from 127.0.0.1:41954 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 06:41:50.112010 14541 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998482572191-14541-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.112838 14541 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:50.113608 14541 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:41:50.120213 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:50.114092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.14.51.65:39649)
0806 06:41:50.114139 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:50.114141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:50.114234 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:41:50.114304 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:50.114305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:50.114313 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:50.114360 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:50.114704 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.115199 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.115834 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:50.116014 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:50.116085 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:50.117865 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:50.117866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:50.117866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:50.117873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:50.118936 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.119408 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.119409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.119410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.119464 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:50.119711 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:50.119712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:50.119712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:50.119828 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:50.119920 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:50.120023 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:41:50.120093 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:41:50.120728 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20396
I20260806 06:41:50.126922 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20535
I20260806 06:41:50.132082 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20677
I20260806 06:41:50.137297 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20301
2026-08-06T06:41:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1868 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20832](info): set up 2 sockets
Aug 06 06:41:50 dist-test-slave-1xrh krb5kdc[20832](info): commencing operation
krb5kdc: starting...
W20260806 06:41:51.206552 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab.
Entry for principal oryx/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab.
I20260806 06:41:51.323923 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43891
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab
--principal=oryx/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:51.402380 20845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:51.402568 20845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:51.402590 20845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:51.403980 20845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:51.404027 20845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:51.404042 20845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:51.404055 20845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:51.404117 20845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:51.405593 20845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab
--principal=oryx/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:51.405844 20845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:51.406075 20845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:51.410815 20850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.410782 20851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.410781 20853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:51.410961 20845 server_base.cc:1034] running on GCE node
I20260806 06:41:51.411365 20845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:51.411676 20845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:51.412842 20845 hybrid_clock.cc:648] HybridClock initialized: now 1785998511412797 us; error 45 us; skew 500 ppm
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:51.419219 20845 init.cc:377] Logged in from keytab as oryx/127.14.51.126@KRBTEST.COM (short username oryx)
I20260806 06:41:51.419648 20845 webserver.cc:460] Webserver started at http://127.14.51.126:37293/ using document root <none> and password file <none>
I20260806 06:41:51.419823 20845 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:51.419862 20845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:51.419946 20845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:51.420948 20845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "9c0c4627528642fcaf93f3f4a7f002c3"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "656ba834b3df4d8fb7c58c634688e368"
server_key_iv: "c5df2831adcafef0208ae789e18aae25"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.421242 20845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "9c0c4627528642fcaf93f3f4a7f002c3"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "656ba834b3df4d8fb7c58c634688e368"
server_key_iv: "c5df2831adcafef0208ae789e18aae25"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.422437 20845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:51.423108 20861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.423353 20845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:51.423439 20845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "9c0c4627528642fcaf93f3f4a7f002c3"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "656ba834b3df4d8fb7c58c634688e368"
server_key_iv: "c5df2831adcafef0208ae789e18aae25"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.423511 20845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:51.439764 20845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:51.441673 20845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:51.441825 20845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:51.445830 20845 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:43891
I20260806 06:41:51.445851 20914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:43891 every 8 connection(s)
I20260806 06:41:51.446211 20845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:51.447156 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:51.448127 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20845
I20260806 06:41:51.448212 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:51.448433 14541 external_mini_cluster.cc:1428] Setting key 4f41821e99f567a59defa6496ca2c942
I20260806 06:41:51.449602 20915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Bootstrap starting.
I20260806 06:41:51.450191 20915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:51.450572 20915 log.cc:824] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:51.451360 20915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: No bootstrap required, opened a new log
I20260806 06:41:51.453552 20915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } }
I20260806 06:41:51.453728 20915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:51.453771 20915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0c4627528642fcaf93f3f4a7f002c3, State: Initialized, Role: FOLLOWER
I20260806 06:41:51.453910 20915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } }
I20260806 06:41:51.454013 20915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:51.454038 20915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:51.454121 20915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:51.454627 20915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } }
I20260806 06:41:51.454710 20915 leader_election.cc:302] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c0c4627528642fcaf93f3f4a7f002c3; no voters:
I20260806 06:41:51.454913 20915 leader_election.cc:288] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:51.454960 20922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:51.455124 20922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9c0c4627528642fcaf93f3f4a7f002c3, State: Running, Role: LEADER
I20260806 06:41:51.455183 20915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:51.455238 20922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } }
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:51.455519 20922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0c4627528642fcaf93f3f4a7f002c3. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } }
I20260806 06:41:51.455612 20922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:51.455530 20924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } }
I20260806 06:41:51.455754 20924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:51.455861 20928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:51.456332 20928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:51.459656 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.448945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.14.51.126:43891)
0806 06:41:51.449141 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:51.449149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:51.449173 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:41:51.449546 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:51.449551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:51.449593 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:51.449733 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:51.450416 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.451816 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.452559 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.453244 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.453349 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.456170 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:51.456174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:51.456174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:51.456184 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:51.458015 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.458440 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.458442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.458442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.458532 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.458748 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.458748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.458749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.458988 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:51.459082 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:51.459298 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 06:41:51.459447 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260806 06:41:51.459957 20939 mvcc.cc:205] Tried to move back new op lower bound from 7315449902929264640 to 7315449902921027584. Current Snapshot: MvccSnapshot[applied={T|T < 7315449902929264640}]
I20260806 06:41:51.460376 20928 catalog_manager.cc:1347] Generated new cluster ID: 48db753e8f4644bf9fdfbd2f9cb1b101
I20260806 06:41:51.460423 20928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:51.469586 20928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:51.470228 20928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:51.479269 20928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Generated new TSK 0
I20260806 06:41:51.479423 20928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
I20260806 06:41:51.509840 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:51.588903 20944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:51.589123 20944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:51.589161 20944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:51.590245 20944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:51.590310 20944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:51.590353 20944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:51.591792 20944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:51.592046 20944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:51.592258 20944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:51.596928 20950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.596923 20952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:51.597092 20944 server_base.cc:1034] running on GCE node
W20260806 06:41:51.596920 20949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:51.597326 20944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:51.597563 20944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:51.598724 20944 hybrid_clock.cc:648] HybridClock initialized: now 1785998511598712 us; error 26 us; skew 500 ppm
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:51.601328 20944 init.cc:377] Logged in from keytab as oryx/127.14.51.65@KRBTEST.COM (short username oryx)
I20260806 06:41:51.601730 20944 webserver.cc:460] Webserver started at http://127.14.51.65:38483/ using document root <none> and password file <none>
I20260806 06:41:51.601946 20944 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:51.601991 20944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:51.602131 20944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:51.603108 20944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "53dd49b30028417f8c5d52c954470f2e"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "b5233d0ed5253d741cc155bf5f0261df"
server_key_iv: "606e80ed5baa61c34d73d08192283e61"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.603523 20944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "53dd49b30028417f8c5d52c954470f2e"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "b5233d0ed5253d741cc155bf5f0261df"
server_key_iv: "606e80ed5baa61c34d73d08192283e61"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.604924 20944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:41:51.605626 20959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.605801 20944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:51.605921 20944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "53dd49b30028417f8c5d52c954470f2e"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "b5233d0ed5253d741cc155bf5f0261df"
server_key_iv: "606e80ed5baa61c34d73d08192283e61"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.606009 20944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:51.618517 20944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:51.620695 20944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:51.620847 20944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:51.621088 20944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:51.621405 20944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:51.621439 20944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.621459 20944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:51.621505 20944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.627435 20944 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:43833
I20260806 06:41:51.627532 21074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:43833 every 8 connection(s)
I20260806 06:41:51.627825 20944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:51.634370 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 20944
I20260806 06:41:51.634501 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:51.634713 14541 external_mini_cluster.cc:1428] Setting key 9f091724ff0f175e36eb7f9575284bf5
I20260806 06:41:51.636837 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.628622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:34061 (local address 127.14.51.126:43891)
0806 06:41:51.628686 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:51.628687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:51.628865 (+ 178us) server_negotiation.cc:407] Connection header received
0806 06:41:51.629340 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:51.629341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:51.629350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:51.629412 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:51.630286 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.630815 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.631818 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.632035 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.632091 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.633979 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:51.633980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:51.633981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:51.633987 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:51.635463 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.635842 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.635843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.635844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.635908 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.636146 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.636147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.636148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.636291 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:51.636360 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:51.636602 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 06:41:51.636696 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:41:51.636842 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.628584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:34061)
0806 06:41:51.628745 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:51.628784 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:51.629244 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:51.629485 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:51.629488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:51.630026 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:51.630205 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.630208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.630896 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.630897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.631746 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.631748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.632109 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.632110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.632202 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.632622 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:51.632630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:51.633888 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:51.635541 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.635546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.635547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.635770 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.635987 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.635988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.635988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.636079 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.636369 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:51.636373 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:51.636477 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 06:41:51.636556 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:41:51.636662 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260806 06:41:51.637616 21075 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:51.637717 21075 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:51.637930 21075 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:51.638397 20878 ts_manager.cc:194] Registered new tserver with Master: 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
I20260806 06:41:51.638993 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:34061
WARNING: no policy specified for oryx/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
I20260806 06:41:51.659965 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:51.739080 21080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:51.739290 21080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:51.739312 21080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:51.740442 21080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:51.740484 21080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:51.740510 21080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:51.741982 21080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:51.742208 21080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:51.742432 21080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:51.747218 21086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.747232 21089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.747247 21087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:51.747507 21080 server_base.cc:1034] running on GCE node
I20260806 06:41:51.747808 21080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:51.748042 21080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:51.749204 21080 hybrid_clock.cc:648] HybridClock initialized: now 1785998511749178 us; error 56 us; skew 500 ppm
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:51.751621 21080 init.cc:377] Logged in from keytab as oryx/127.14.51.66@KRBTEST.COM (short username oryx)
I20260806 06:41:51.752061 21080 webserver.cc:460] Webserver started at http://127.14.51.66:38655/ using document root <none> and password file <none>
I20260806 06:41:51.752280 21080 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:51.752350 21080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:51.752480 21080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:51.753451 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "d2b840b3392e4f8cac58383a1d003743"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "50ee099518e3a93cb969b7134d0c7dfc"
server_key_iv: "de7ad180d466c70ffc3fff6ac7cd811f"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.753821 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "d2b840b3392e4f8cac58383a1d003743"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "50ee099518e3a93cb969b7134d0c7dfc"
server_key_iv: "de7ad180d466c70ffc3fff6ac7cd811f"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.755187 21080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:51.756013 21096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.756189 21080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:51.756282 21080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "d2b840b3392e4f8cac58383a1d003743"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "50ee099518e3a93cb969b7134d0c7dfc"
server_key_iv: "de7ad180d466c70ffc3fff6ac7cd811f"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.756349 21080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:51.774646 21080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:51.776362 21080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:51.776471 21080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:51.776628 21080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:51.776863 21080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:51.776893 21080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.776913 21080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:51.776925 21080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.782335 21080 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:42405
I20260806 06:41:51.782413 21216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:42405 every 8 connection(s)
I20260806 06:41:51.782703 21080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:51.784010 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21080
I20260806 06:41:51.784087 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:51.784274 14541 external_mini_cluster.cc:1428] Setting key 7ac423bf32c9831693439d39672657d6
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:51.792634 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.783445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:38675 (local address 127.14.51.126:43891)
0806 06:41:51.783514 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:51.783516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:51.783682 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:41:51.784234 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:51.784235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:51.784247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:51.784303 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:51.785379 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.785874 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.786890 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.787098 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.787212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.789731 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:51.789732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:51.789732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:51.789738 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:51.791114 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.791681 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.791682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.791682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.791733 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.792015 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.792015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.792015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.792112 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:51.792161 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:51.792491 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 06:41:51.792572 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:41:51.792861 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.783397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:38675)
0806 06:41:51.783557 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:51.783594 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:51.784118 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:51.784367 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:51.784371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:51.785074 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:51.785271 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.785276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.785986 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.785987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.786789 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.786791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.787223 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.787224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.787347 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.787855 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:51.787862 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:51.789626 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:51.791242 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.791249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.791252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.791597 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.791801 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.791802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.791803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.791947 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.792176 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:51.792182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:51.792341 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 06:41:51.792480 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 06:41:51.792658 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for oryx/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:41:51.793329 21217 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:51.793430 21217 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:51.793592 21217 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:51.793946 20878 ts_manager.cc:194] Registered new tserver with Master: d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:51.794319 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:38675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
I20260806 06:41:51.809863 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:51.888131 21223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:51.888350 21223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:51.888387 21223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:51.889500 21223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:51.889564 21223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:51.889607 21223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:51.890964 21223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:51.891227 21223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:51.891436 21223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:51.896335 21229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.896335 21230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:51.896349 21232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:51.896644 21223 server_base.cc:1034] running on GCE node
I20260806 06:41:51.896881 21223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:51.897084 21223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:51.898218 21223 hybrid_clock.cc:648] HybridClock initialized: now 1785998511898195 us; error 38 us; skew 500 ppm
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:51.900578 21223 init.cc:377] Logged in from keytab as oryx/127.14.51.67@KRBTEST.COM (short username oryx)
I20260806 06:41:51.900990 21223 webserver.cc:460] Webserver started at http://127.14.51.67:42095/ using document root <none> and password file <none>
I20260806 06:41:51.901245 21223 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:51.901321 21223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:51.901485 21223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:51.902552 21223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "09a149d9783b4f26b1282f2f38a8ac6a"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "7a0477738f71046c0ad78e833aefc4bb"
server_key_iv: "3c08b756934229337e103e0f35b329bc"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.902910 21223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "09a149d9783b4f26b1282f2f38a8ac6a"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "7a0477738f71046c0ad78e833aefc4bb"
server_key_iv: "3c08b756934229337e103e0f35b329bc"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.904261 21223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:51.904947 21239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.905128 21223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:51.905220 21223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "09a149d9783b4f26b1282f2f38a8ac6a"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "7a0477738f71046c0ad78e833aefc4bb"
server_key_iv: "3c08b756934229337e103e0f35b329bc"
server_key_version: "encryptionkey@0"
I20260806 06:41:51.905298 21223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:51.922596 21223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:51.924437 21223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:51.924588 21223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:51.924796 21223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:51.925072 21223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:51.925125 21223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.925164 21223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:51.925194 21223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:51.931478 21223 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37333
I20260806 06:41:51.931514 21359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37333 every 8 connection(s)
I20260806 06:41:51.931818 21223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:51.934464 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21223
I20260806 06:41:51.934576 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:51.934765 14541 external_mini_cluster.cc:1428] Setting key 502e5d59a55b2e4620fda4a910c5ee91
Aug 06 06:41:51 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:51.940327 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.932549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:47429 (local address 127.14.51.126:43891)
0806 06:41:51.932614 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:51.932616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:51.932782 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:41:51.933243 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:51.933250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:51.933261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:51.933303 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:51.934139 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.934756 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.936039 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.936217 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.936272 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.938210 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:51.938211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:51.938211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:51.938218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:51.939298 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.939625 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.939626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.939626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.939671 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:51.939863 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:51.939863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:51.939863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:51.939949 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:51.939993 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:51.940189 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:41:51.940266 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:41:51.940615 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:51.932509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:47429)
0806 06:41:51.932663 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:51.932701 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:51.933149 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:51.933351 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:51.933354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:51.933892 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:51.934067 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.934072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.934869 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.934870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.935967 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:51.935970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:51.936284 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:51.936285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:51.936373 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:51.936758 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:51.936766 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:51.938128 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:51.939350 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.939360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.939361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.939568 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.939725 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:51.939726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:51.939726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:51.939810 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:51.939999 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:51.940002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:51.940094 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 06:41:51.940166 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:41:51.940292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 06:41:51.941039 21360 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:51.941138 21360 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:51.941316 21360 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:51.941663 20878 ts_manager.cc:194] Registered new tserver with Master: 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333)
I20260806 06:41:51.942080 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:47429
I20260806 06:41:51.946468 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.51.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.125.keytab.
Entry for principal oryx/127.14.51.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.125.keytab.
I20260806 06:41:51.971465 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:41275
--webserver_interface=127.14.51.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.125.keytab
--principal=oryx/127.14.51.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43891,127.14.51.125:41275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:52.050987 21366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:52.051211 21366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:52.051236 21366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:52.052373 21366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:52.052413 21366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:52.052428 21366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:52.052475 21366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:52.052502 21366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:52.053918 21366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43891,127.14.51.125:41275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.125.keytab
--principal=oryx/127.14.51.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:41275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.51.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
W20260806 06:41:52.054028 21366 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.51.126:43891,127.14.51.125:41275'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 06:41:52.054157 21366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:52.054360 21366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.059074 21373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.059238 21366 server_base.cc:1034] running on GCE node
W20260806 06:41:52.059082 21372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.059270 21375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.059537 21366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.059810 21366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.060969 21366 hybrid_clock.cc:648] HybridClock initialized: now 1785998512060954 us; error 24 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.063367 21366 init.cc:377] Logged in from keytab as oryx/127.14.51.125@KRBTEST.COM (short username oryx)
I20260806 06:41:52.063719 21366 webserver.cc:460] Webserver started at http://127.14.51.125:39079/ using document root <none> and password file <none>
I20260806 06:41:52.063932 21366 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.063972 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.064059 21366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:52.064972 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/instance:
uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "99765c137ecd65bff3d222fd84ae9e5b"
server_key_iv: "2d2b90021ed37a5fe6dc506a07d5e442"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.065258 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal/instance:
uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "99765c137ecd65bff3d222fd84ae9e5b"
server_key_iv: "2d2b90021ed37a5fe6dc506a07d5e442"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.066543 21366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:41:52.067224 21383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.067414 21366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:52.067502 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "99765c137ecd65bff3d222fd84ae9e5b"
server_key_iv: "2d2b90021ed37a5fe6dc506a07d5e442"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.067552 21366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.088186 21366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.089895 21366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:52.089998 21366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.093923 21366 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.125:41275
I20260806 06:41:52.093969 21438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.125:41275 every 8 connection(s)
I20260806 06:41:52.094277 21366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
I20260806 06:41:52.095247 21439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:52.095856 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21366
I20260806 06:41:52.096987 21439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.125@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
I20260806 06:41:52.106189 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.097767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.14.51.126:43891)
0806 06:41:52.097838 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.097840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.097998 (+ 158us) server_negotiation.cc:407] Connection header received
0806 06:41:52.098069 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.098070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.098082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.098155 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.098574 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.099330 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.100209 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.100422 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.100498 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.102761 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.102763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.102763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.102770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.104608 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.105059 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.105060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.105061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.105122 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.105430 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.105431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.105431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.105631 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.105695 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.106002 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.106085 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:41:52.106485 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.097700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44882)
0806 06:41:52.097891 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.097933 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.097998 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.098166 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.098168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.098388 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.098486 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.098490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.099423 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.099424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.100095 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.100097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.100503 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.100505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.100585 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.101205 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.101230 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.102672 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.104694 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.104700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.104701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.104983 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.105204 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.105204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.105205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.105355 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.105707 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.105714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.105869 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.105956 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.106102 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260806 06:41:52.106608 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.096466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.14.51.125:41275)
0806 06:41:52.096644 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.096650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.096673 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:41:52.097080 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.097086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.097129 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.097237 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.098022 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.099253 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.100002 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.100339 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.100495 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.103401 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.103408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.103409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.103414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.105181 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.105565 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.105569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.105569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.105623 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.105846 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.105846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.105847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.106007 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.106157 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.106390 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.106495 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260806 06:41:52.107265 21439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.125@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
I20260806 06:41:52.113714 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.107541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33074)
0806 06:41:52.107591 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.107602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.107638 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.107764 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.107764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.107934 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.107974 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.107975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.108660 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.108661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.109198 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.109200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.109601 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.109602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.109663 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.110063 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.110068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.111209 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.112736 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.112737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.112737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.112891 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.113118 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.113118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.113119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.113172 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.113410 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.113410 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.113445 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.113511 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.113611 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:41:52.113770 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.107591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.14.51.125:41275)
0806 06:41:52.107621 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.107623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.107645 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:41:52.107698 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.107698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.107708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.107760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.108068 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.108585 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.109276 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.109514 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.109602 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.111310 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.111311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.111311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.111317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.112637 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.112963 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.112964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.112964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.113040 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.113244 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.113245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.113245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.113345 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.113417 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.113538 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.113660 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":16}
I20260806 06:41:52.114976 21439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Bootstrap starting.
I20260806 06:41:52.115782 21439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:52.116003 21439 log.cc:824] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:52.116664 21439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: No bootstrap required, opened a new log
I20260806 06:41:52.117959 21439 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } }
I20260806 06:41:52.118093 21439 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:52.118115 21439 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3dc4b71b1004aecbba94ae63fcad0fa, State: Initialized, Role: FOLLOWER
I20260806 06:41:52.118196 21439 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } }
I20260806 06:41:52.118412 21439 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:52.118397 21446 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } }
I20260806 06:41:52.118507 21446 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:41:52.119956 21459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:41:52.120018 21459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:41:52.121651 21366 master_runner.cc:428] Detected that this master f3dc4b71b1004aecbba94ae63fcad0fa is joining an existing cluster
I20260806 06:41:52.121726 21366 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.51.125:41275
I20260806 06:41:52.129210 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.122689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33090)
0806 06:41:52.122891 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.122899 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.123238 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.123399 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.123400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.123602 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.123663 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.123665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.124340 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.124341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.124858 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.124859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.125303 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.125304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.125368 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.125837 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.125841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.126260 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.128197 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.128198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.128366 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.128574 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.128576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.128629 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.128868 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.128869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.128905 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.128973 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.129079 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":72,"threads_started":1}
I20260806 06:41:52.129268 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.122824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.14.51.125:41275)
0806 06:41:52.122853 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.122855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.122942 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:41:52.123316 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.123316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.123325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.123351 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.123744 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.124255 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.124950 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.125215 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.125275 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.126343 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.126344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.126345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.126350 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.128095 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128454 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.128455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.128455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.128503 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128699 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.128700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.128700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.128804 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.128877 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.128997 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.129143 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":15}
I20260806 06:41:52.129663 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.122671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.14.51.126:43891)
0806 06:41:52.122775 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.122777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.122824 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:41:52.122896 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.122897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.122906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.122950 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.123336 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.123894 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.124994 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.125281 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.125337 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.126599 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.126600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.126601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.126608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.128397 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128817 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.128818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.128818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.128876 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.129061 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.129062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.129062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.129194 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.129266 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.129474 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.129563 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 06:41:52.129976 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.122637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44892)
0806 06:41:52.122768 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.122779 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.122839 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.122952 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.122953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.123177 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.123260 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.123262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.123978 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.123979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.124880 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.124883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.125388 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.125389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.125438 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.126049 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.126053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.126504 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.128493 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.128495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.128727 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.128949 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.128950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.128950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.129005 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.129314 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.129315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.129356 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.129434 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.129874 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.126@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
I20260806 06:41:52.139467 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.132395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.14.51.125:41275)
0806 06:41:52.132431 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.132433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.132586 (+ 153us) server_negotiation.cc:407] Connection header received
0806 06:41:52.132666 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.132666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.132674 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.132719 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.133183 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.133600 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.134714 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.134882 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.134936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.136988 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.136989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.136989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.136996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.138279 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.138672 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.138674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.138674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.138730 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.138941 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.138942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.138942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.139039 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.139112 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.139310 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.139381 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:52.139520 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.132300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33092)
0806 06:41:52.132467 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.132508 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.132594 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.132726 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.132729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.133052 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.133109 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.133111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.133675 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.133676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.134630 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.134633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.134959 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.134960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.135039 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.135429 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.135434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.136881 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.138382 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.138389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.138391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.138591 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.138797 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.138798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.138799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.138856 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.139106 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.139113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.139184 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.139284 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.139392 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260806 06:41:52.140002 20878 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.51.125:41275: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
member_type: NON_VOTER
last_known_addr {
host: "127.14.51.125"
port: 41275
}
attrs {
promote: true
}
}
dest_uuid: "9c0c4627528642fcaf93f3f4a7f002c3"
cas_config_opid_index: -1
I20260806 06:41:52.140327 20878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } }
I20260806 06:41:52.140784 21471 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.141162 21411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:52.141876 21471 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.51.125:41275
I20260806 06:41:52.142038 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.142131 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:52.142102 21411 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 06:41:52.142410 21366 master.cc:537] Master@127.14.51.125:41275 shutting down...
I20260806 06:41:52.142617 21469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:41:52.143309 20862 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 06:41:52.144048 20862 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 -> Peer f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125:41275): Couldn't send request to peer f3dc4b71b1004aecbba94ae63fcad0fa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:41:52.144454 21446 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:52.144542 21446 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } }
I20260806 06:41:52.144780 21446 leader_election.cc:288] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891)
I20260806 06:41:52.145015 20888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" is_pre_election: true
I20260806 06:41:52.145336 21385 leader_election.cc:302] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f3dc4b71b1004aecbba94ae63fcad0fa; no voters: 9c0c4627528642fcaf93f3f4a7f002c3
I20260806 06:41:52.145468 21446 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:41:52.145675 21366 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:52.145744 21366 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:52.145776 21366 tablet_replica.cc:330] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: stopping tablet replica
I20260806 06:41:52.158339 21366 master.cc:552] Master@127.14.51.125:41275 shutdown complete.
I20260806 06:41:52.158387 21366 master_runner.cc:305] Clearing existing system tablet
I20260806 06:41:52.158533 21366 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.158572 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.159250 21366 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:52.159785 21475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.159945 21366 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.160015 21366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.160045 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "99765c137ecd65bff3d222fd84ae9e5b"
server_key_iv: "2d2b90021ed37a5fe6dc506a07d5e442"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.161026 21366 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:52.162436 21366 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:41:52.162511 21366 log.cc:1196] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 06:41:52.162678 21366 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Deleting consensus metadata
I20260806 06:41:52.162794 21366 master_runner.cc:315] Copying system tablet from 127.14.51.126:43891
I20260806 06:41:52.163482 21366 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: tablet copy: Beginning tablet copy session from remote peer at address 127.14.51.126:43891
I20260806 06:41:52.169343 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.163900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.14.51.126:43891)
0806 06:41:52.163950 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.163953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.164005 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:41:52.164082 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.164083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.164092 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.164136 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.164462 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.164949 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.165662 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.165866 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.165955 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.166878 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.166879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.166879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.166886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.168227 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.168573 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.168574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.168574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.168632 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.168834 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.168835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.168835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.168943 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.169019 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.169189 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.169257 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:41:52.169359 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.163830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44900)
0806 06:41:52.163946 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.163958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.164023 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.164158 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.164159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.164322 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.164370 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.164372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.165037 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.165038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.165580 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.165581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.165955 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.165956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.166030 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.166500 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.166505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.166770 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.168325 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.168326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.168327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.168500 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.168699 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.168700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.168700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.168756 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.169011 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.169011 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.169055 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.169151 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.169254 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260806 06:41:52.169930 20899 tablet_copy_service.cc:140] P 9c0c4627528642fcaf93f3f4a7f002c3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f3dc4b71b1004aecbba94ae63fcad0fa ({username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:44900)
I20260806 06:41:52.170022 20899 tablet_copy_service.cc:161] P 9c0c4627528642fcaf93f3f4a7f002c3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f3dc4b71b1004aecbba94ae63fcad0fa at {username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:44900: session id = f3dc4b71b1004aecbba94ae63fcad0fa-00000000000000000000000000000000
I20260806 06:41:52.170714 20899 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:41:52.171366 21366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:41:52.172262 21366 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: tablet copy: Starting download of 0 data blocks...
I20260806 06:41:52.172408 21366 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: tablet copy: Starting download of 1 WAL segments...
I20260806 06:41:52.173259 21366 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:41:52.174010 20899 tablet_copy_service.cc:342] P 9c0c4627528642fcaf93f3f4a7f002c3: Request end of tablet copy session f3dc4b71b1004aecbba94ae63fcad0fa-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:44900
I20260806 06:41:52.174068 20899 tablet_copy_service.cc:433] P 9c0c4627528642fcaf93f3f4a7f002c3: ending tablet copy session f3dc4b71b1004aecbba94ae63fcad0fa-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f3dc4b71b1004aecbba94ae63fcad0fa
I20260806 06:41:52.176498 21366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.177634 21484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.177708 21482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.179994 21486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.180114 21366 server_base.cc:1034] running on GCE node
I20260806 06:41:52.180223 21366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.180370 21366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.181505 21366 hybrid_clock.cc:648] HybridClock initialized: now 1785998512181498 us; error 23 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.182773 21366 init.cc:377] Logged in from keytab as oryx/127.14.51.125@KRBTEST.COM (short username oryx)
I20260806 06:41:52.183058 21366 webserver.cc:460] Webserver started at http://127.14.51.125:44293/ using document root <none> and password file <none>
I20260806 06:41:52.183221 21366 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.183259 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.183830 21366 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:52.184378 21493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.184513 21366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:52.184559 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "99765c137ecd65bff3d222fd84ae9e5b"
server_key_iv: "2d2b90021ed37a5fe6dc506a07d5e442"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.184608 21366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.199474 21366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.200982 21366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.204011 21549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.125:41275 every 8 connection(s)
I20260806 06:41:52.203994 21366 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.125:41275
I20260806 06:41:52.204325 21366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
I20260806 06:41:52.205180 21550 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:41:52.205713 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Bootstrap starting.
I20260806 06:41:52.209185 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:41:52.209450 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Bootstrap complete.
I20260806 06:41:52.209697 21550 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } }
I20260806 06:41:52.209786 21550 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f3dc4b71b1004aecbba94ae63fcad0fa, State: Initialized, Role: LEARNER
I20260806 06:41:52.209827 21550 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } }
I20260806 06:41:52.210000 21550 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:52.210006 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.210213 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: LEARNER
I20260806 06:41:52.211488 21566 catalog_manager.cc:1259] Loaded cluster ID: 48db753e8f4644bf9fdfbd2f9cb1b101
I20260806 06:41:52.211818 21566 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: loading cluster ID for follower catalog manager: success
I20260806 06:41:52.213408 21566 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: acquiring CA information for follower catalog manager: success
I20260806 06:41:52.213757 21566 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:41:52.215873 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.208129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.14.51.125:41275)
0806 06:41:52.208244 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.208249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.208258 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:52.208277 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.208278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.208287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.208368 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.208958 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.209759 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.210779 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.211021 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.211357 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.212601 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.212603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.212603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.212609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.214726 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.215090 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.215091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.215091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.215170 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.215377 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.215378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.215379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.215486 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.215587 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.215696 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.215779 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.14.51.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.124.keytab.
Entry for principal oryx/127.14.51.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.124.keytab.
I20260806 06:41:52.240193 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:33857
--webserver_interface=127.14.51.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.124.keytab
--principal=oryx/127.14.51.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:52.330940 21570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:52.331147 21570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:52.331171 21570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:52.332265 21570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:52.332297 21570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:52.332310 21570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:52.332324 21570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:52.332335 21570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:52.333714 21570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.124.keytab
--principal=oryx/127.14.51.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:33857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.51.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:52.333895 21570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:52.334093 21570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:41:52.338666 21570 server_base.cc:1034] running on GCE node
W20260806 06:41:52.338632 21578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.338646 21576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.338618 21575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.339107 21570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.339484 21570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.340634 21570 hybrid_clock.cc:648] HybridClock initialized: now 1785998512340615 us; error 36 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.343219 21570 init.cc:377] Logged in from keytab as oryx/127.14.51.124@KRBTEST.COM (short username oryx)
I20260806 06:41:52.343685 21570 webserver.cc:460] Webserver started at http://127.14.51.124:40581/ using document root <none> and password file <none>
I20260806 06:41:52.343884 21570 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.343930 21570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.344015 21570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:52.345012 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/instance:
uuid: "f3a86a40006a4549bb96ad7e77b671a2"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "b9a73f539e628f7789498020af300313"
server_key_iv: "2bbe9916055b8a1cbe1f81f138f2d9ff"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.345304 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal/instance:
uuid: "f3a86a40006a4549bb96ad7e77b671a2"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "b9a73f539e628f7789498020af300313"
server_key_iv: "2bbe9916055b8a1cbe1f81f138f2d9ff"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.346498 21570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:41:52.347241 21586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.347419 21570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:52.347517 21570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "f3a86a40006a4549bb96ad7e77b671a2"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "b9a73f539e628f7789498020af300313"
server_key_iv: "2bbe9916055b8a1cbe1f81f138f2d9ff"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.347600 21570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.363603 21570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.365443 21570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:52.365636 21570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.372175 21570 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.124:33857
I20260806 06:41:52.372226 21639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.124:33857 every 8 connection(s)
I20260806 06:41:52.372591 21570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
I20260806 06:41:52.373745 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:52.374723 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21570
I20260806 06:41:52.376114 21640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.124@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:52.385789 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.375494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.14.51.124:33857)
0806 06:41:52.375799 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.375806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.375862 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:41:52.376241 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.376247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.376290 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.376443 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.377208 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.378140 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.379156 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.379708 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.379812 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.382117 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.382124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.382125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.382131 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.384182 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.384570 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.384573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.384573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.384668 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.384882 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.384883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.384883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.385109 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.385244 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.385354 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.385496 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":259,"thread_start_us":215,"threads_started":1}
I20260806 06:41:52.385851 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.376920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44904)
0806 06:41:52.377127 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.377167 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.377349 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.377528 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.377531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.377816 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.377940 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.377945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.378935 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.378936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.379861 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.379863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.380317 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.380318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.380366 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.380801 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.380808 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.382525 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.384575 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.384580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.384582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.384826 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.385072 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.385072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.385073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.385125 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.385373 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.385379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.385439 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.385574 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.385694 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260806 06:41:52.385869 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.377168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.14.51.126:43891)
0806 06:41:52.377256 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.377265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.377328 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:41:52.377414 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.377415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.377432 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.377490 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.378038 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.378791 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.379943 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.380211 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.380279 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.382634 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.382635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.382636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.382644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.384181 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.384919 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.384920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.384921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.384982 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.385190 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.385191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.385191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.385303 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.385381 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.385686 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.385763 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:41:52.386551 21640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.124@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
I20260806 06:41:52.392422 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.386810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33120)
0806 06:41:52.386856 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.386866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.386901 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.387010 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.387010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.387175 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.387285 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.387286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.387932 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.387932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.388403 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.388405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.388717 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.388718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.388799 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.389236 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.389239 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.390316 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.391676 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.391676 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.391677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.391829 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.391997 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.391998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.391998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.392054 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.392235 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.392236 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.392265 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.392308 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.392369 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:41:52.392452 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.386870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.14.51.125:41275)
0806 06:41:52.386915 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.386917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.386925 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:52.386948 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.386949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.386965 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.387009 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.387352 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.387863 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.388470 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.388657 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.388706 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.390393 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.390395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.390395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.390411 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.391608 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.391892 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.391892 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.391893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.391942 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.392091 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.392091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.392091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.392176 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.392232 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.392329 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.392388 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:52.393098 21640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.124@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
I20260806 06:41:52.398396 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.393379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.0.0.1:44058)
0806 06:41:52.393426 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.393437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.393480 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.393580 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.393581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.393681 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.393705 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.393706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.394275 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.394276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.394709 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.394710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.395024 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.395025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.395063 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.395477 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.395486 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.396443 (+ 957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.397638 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.397639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.397639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.397791 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.397966 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.397967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.397967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.398013 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.398194 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.398194 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.398224 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.398273 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.398341 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:41:52.398458 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.393434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.14.51.124:33857)
0806 06:41:52.393482 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.393484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.393511 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:41:52.393534 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.393535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.393541 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.393577 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.393759 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.394210 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.394774 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.394967 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.395011 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.396511 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.396512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.396512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.396517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.397572 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.397855 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.397856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.397856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.397910 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.398066 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.398067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.398067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.398145 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.398188 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.398286 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.398384 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":31}
I20260806 06:41:52.399765 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Bootstrap starting.
I20260806 06:41:52.400800 21640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:52.401309 21640 log.cc:824] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:52.402062 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: No bootstrap required, opened a new log
I20260806 06:41:52.403992 21640 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } }
I20260806 06:41:52.404140 21640 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:52.404188 21640 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a86a40006a4549bb96ad7e77b671a2, State: Initialized, Role: FOLLOWER
I20260806 06:41:52.404337 21640 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } }
I20260806 06:41:52.404656 21645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } } }
I20260806 06:41:52.404726 21645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:52.404701 21640 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 06:41:52.406754 21657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:41:52.406813 21657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:41:52.409116 21570 master_runner.cc:428] Detected that this master f3a86a40006a4549bb96ad7e77b671a2 is joining an existing cluster
I20260806 06:41:52.409164 21570 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.51.124:33857
I20260806 06:41:52.418431 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.410645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33122 (local address 127.14.51.125:41275)
0806 06:41:52.410683 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.410685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.411681 (+ 996us) server_negotiation.cc:407] Connection header received
0806 06:41:52.411777 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.411778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.411788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.411816 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.412312 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.413031 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414205 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414615 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414670 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.415964 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.415966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.415966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.415971 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.417333 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.417735 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.417736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.417736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.417782 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.417977 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.417978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.417978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.418065 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.418112 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.418298 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.418353 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:52.418681 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.411154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33122)
0806 06:41:52.411511 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.411521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.411707 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.411878 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.411879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.412126 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.412221 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.412223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.413161 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.413161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414080 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.414082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414750 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414841 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.415357 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.415362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.415819 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.417418 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.417419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.417419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.417660 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.417840 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.417841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.417841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.417903 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.418131 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.418132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.418178 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.418277 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.418373 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":4,"thread_start_us":258,"threads_started":1}
I20260806 06:41:52.435453 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.411178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.14.51.126:43891)
0806 06:41:52.411220 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.411222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.411233 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:52.411321 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.411321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.411362 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.411448 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.412095 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.412913 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414289 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414601 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414639 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.432589 (+ 17950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.432590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.432590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.432598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.434139 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434663 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.434664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.434664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.434720 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434903 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.434904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.434904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.435018 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.435103 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.435295 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.435366 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:52.435516 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.410834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44908)
0806 06:41:52.411159 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.411170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.411247 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.411523 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.411524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.411854 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.412006 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.412009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.412991 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.412992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414197 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.414200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414742 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414850 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.431840 (+ 16990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.431844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.432500 (+ 656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.434242 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.434243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.434565 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.434795 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.434796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.434855 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.435129 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.435129 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.435191 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.435258 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.435356 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":28,"thread_start_us":175,"threads_started":1}
I20260806 06:41:52.435807 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.411544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.14.51.124:33857)
0806 06:41:52.411611 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.411613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.411996 (+ 383us) server_negotiation.cc:407] Connection header received
0806 06:41:52.412082 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.412082 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.412090 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.412116 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.412607 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.413224 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414285 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414593 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414635 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.432865 (+ 18230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.432866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.432866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.432873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.434503 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434877 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.434879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.434879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.434966 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.435224 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.435224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.435225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.435372 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.435462 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.435646 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.435720 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:41:52.436105 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.411655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.0.0.1:44072)
0806 06:41:52.411826 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.411835 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.412018 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.412159 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.412160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.412348 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.412518 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.412520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.413314 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.413315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414198 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.414200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.414760 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.414760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.414850 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.431736 (+ 16886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.431740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.432493 (+ 753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.434586 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.434587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.434587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.434791 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.435080 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.435081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.435081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.435170 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.435486 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.435486 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.435546 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.435909 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.435989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260806 06:41:52.437757 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20944
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998511, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.126@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
I20260806 06:41:52.445511 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.437628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.14.51.124:33857)
0806 06:41:52.437664 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.437665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.437674 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:52.437708 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.437709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.437718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.437774 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.438123 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.438879 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.439946 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.440223 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.440278 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.442482 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.442483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.442483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.442489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.444206 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.444665 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.444666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.444666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.444723 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.444963 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.444964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.444965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.445096 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.445159 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.445357 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.445428 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:52.445590 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.437574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.0.0.1:44076)
0806 06:41:52.437618 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.437627 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.437665 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.437824 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.437825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.437990 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.438041 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.438042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.438974 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.438975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.439814 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.439817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.440357 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.440358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.440427 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.440865 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.440870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.442375 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.444299 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.444300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.444300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.444577 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.444799 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.444800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.444800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.444880 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.445179 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.445181 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.445230 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.445317 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.445436 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:41:52.448977 20878 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.51.124:33857: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2"
member_type: NON_VOTER
last_known_addr {
host: "127.14.51.124"
port: 33857
}
attrs {
promote: true
}
}
dest_uuid: "9c0c4627528642fcaf93f3f4a7f002c3"
cas_config_opid_index: 5
I20260806 06:41:52.449186 20878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
I20260806 06:41:52.449717 21469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.450719 21469 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.51.124:33857
I20260806 06:41:52.450783 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.450804 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:43833
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:43891
--webserver_port=38483
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:41:52.450847 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:52.450986 21614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:52.452364 21614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:41:52.452664 21666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:52.453416 21570 master.cc:537] Master@127.14.51.124:33857 shutting down...
I20260806 06:41:52.454363 21614 raft_consensus.cc:858] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
W20260806 06:41:52.454443 21614 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
I20260806 06:41:52.455602 21614 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.456858 21671 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.456938 21671 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 06:41:52.456971 21671 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.457007 21671 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 06:41:52.457187 21645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0c4627528642fcaf93f3f4a7f002c3. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } }
I20260806 06:41:52.457268 21645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 06:41:52.457683 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.457785 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:52.458354 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.450977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.14.51.125:41275)
0806 06:41:52.451017 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.451019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.451028 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:52.451052 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.451053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.451074 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.451233 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:52.453248 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.453939 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.457172 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.457757 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.457837 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.457902 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.458223 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.458270 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:41:52.458366 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.450482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33132)
0806 06:41:52.450631 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.450641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.450678 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.453123 (+ 2445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.453124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.453126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:52.453155 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.453156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.454037 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.454038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.457059 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.457063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.458043 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.458044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.458116 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.458139 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.458203 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.458249 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260806 06:41:52.458690 21523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 LEARNER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:41:52.458990 21471 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 06:41:52.459264 21570 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 06:41:52.459357 21570 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 06:41:52.459395 21570 tablet_replica.cc:330] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: stopping tablet replica
W20260806 06:41:52.459995 20862 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 -> Peer f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857): Couldn't send request to peer f3a86a40006a4549bb96ad7e77b671a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:41:52.459954 21523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.461169 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0c4627528642fcaf93f3f4a7f002c3. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.461203 21673 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: NON_VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: true } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.461315 21673 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: LEARNER
I20260806 06:41:52.461267 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: LEARNER
I20260806 06:41:52.463614 21471 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: attempting to promote NON_VOTER f3dc4b71b1004aecbba94ae63fcad0fa to VOTER
I20260806 06:41:52.463831 21471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
I20260806 06:41:52.464486 21523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 LEARNER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260806 06:41:52.464672 20862 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 -> Peer f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857): Couldn't send request to peer f3a86a40006a4549bb96ad7e77b671a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:41:52.464747 21666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 06:41:52.465381 21666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.466593 21523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.467649 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.467749 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:52.468128 21471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.468222 21471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:52.468338 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.468395 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:52.486773 21570 master.cc:552] Master@127.14.51.124:33857 shutdown complete.
I20260806 06:41:52.486827 21570 master_runner.cc:305] Clearing existing system tablet
I20260806 06:41:52.486961 21570 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.487007 21570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.487905 21570 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:52.488502 21679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.488631 21570 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.488683 21570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:52.488709 21570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "f3a86a40006a4549bb96ad7e77b671a2"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "b9a73f539e628f7789498020af300313"
server_key_iv: "2bbe9916055b8a1cbe1f81f138f2d9ff"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.489452 21570 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:52.491142 21570 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:41:52.491223 21570 log.cc:1196] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 06:41:52.491496 21570 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Deleting consensus metadata
I20260806 06:41:52.491647 21570 master_runner.cc:315] Copying system tablet from 127.14.51.126:43891
I20260806 06:41:52.492311 21570 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: tablet copy: Beginning tablet copy session from remote peer at address 127.14.51.126:43891
I20260806 06:41:52.500224 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.492620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:44922)
0806 06:41:52.492754 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.492766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.492846 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.493004 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.493005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.493260 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.493325 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.493328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.494272 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.494273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.495255 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.495257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.495779 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.495781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.495877 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.496410 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.496416 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.496846 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.498905 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.498906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.498907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.499172 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.499418 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.499419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.499419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.499495 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.499851 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.499852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.499904 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.499992 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.500103 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260806 06:41:52.500223 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.492679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.14.51.126:43891)
0806 06:41:52.492724 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.492726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.492835 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:41:52.492916 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.492917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.492929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.492987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.493414 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.494171 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.495349 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.495692 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.495767 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.496954 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.496955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.496956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.496963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.498804 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.499247 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.499249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.499249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.499332 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.499576 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.499577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.499578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.499731 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.499811 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.500048 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.500137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:52.500990 20899 tablet_copy_service.cc:140] P 9c0c4627528642fcaf93f3f4a7f002c3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f3a86a40006a4549bb96ad7e77b671a2 ({username='oryx', principal='oryx/127.14.51.124@KRBTEST.COM'} at 127.0.0.1:44922)
I20260806 06:41:52.501049 20899 tablet_copy_service.cc:161] P 9c0c4627528642fcaf93f3f4a7f002c3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f3a86a40006a4549bb96ad7e77b671a2 at {username='oryx', principal='oryx/127.14.51.124@KRBTEST.COM'} at 127.0.0.1:44922: session id = f3a86a40006a4549bb96ad7e77b671a2-00000000000000000000000000000000
I20260806 06:41:52.501590 20899 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:41:52.502074 21570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:41:52.503561 21570 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: tablet copy: Starting download of 0 data blocks...
I20260806 06:41:52.503715 21570 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: tablet copy: Starting download of 1 WAL segments...
I20260806 06:41:52.504562 21570 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:41:52.505486 20899 tablet_copy_service.cc:342] P 9c0c4627528642fcaf93f3f4a7f002c3: Request end of tablet copy session f3a86a40006a4549bb96ad7e77b671a2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.51.124@KRBTEST.COM'} at 127.0.0.1:44922
I20260806 06:41:52.505539 20899 tablet_copy_service.cc:433] P 9c0c4627528642fcaf93f3f4a7f002c3: ending tablet copy session f3a86a40006a4549bb96ad7e77b671a2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f3a86a40006a4549bb96ad7e77b671a2
I20260806 06:41:52.508150 21570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.509290 21686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.509559 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.509990 21570 server_base.cc:1034] running on GCE node
W20260806 06:41:52.510468 21687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.510654 21570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.510803 21570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.511935 21570 hybrid_clock.cc:648] HybridClock initialized: now 1785998512511945 us; error 48 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.513450 21570 init.cc:377] Logged in from keytab as oryx/127.14.51.124@KRBTEST.COM (short username oryx)
I20260806 06:41:52.513688 21570 webserver.cc:460] Webserver started at http://127.14.51.124:34321/ using document root <none> and password file <none>
I20260806 06:41:52.513799 21570 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.513832 21570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.514564 21570 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:52.515182 21696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.515370 21570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:52.515432 21570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "f3a86a40006a4549bb96ad7e77b671a2"
format_stamp: "Formatted at 2026-08-06 06:41:52 on dist-test-slave-1xrh"
server_key: "b9a73f539e628f7789498020af300313"
server_key_iv: "2bbe9916055b8a1cbe1f81f138f2d9ff"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.515484 21570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.521075 21570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.522994 21570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.525918 21570 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.124:33857
I20260806 06:41:52.526301 21570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
I20260806 06:41:52.526625 21750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.124:33857 every 8 connection(s)
I20260806 06:41:52.527225 21751 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:41:52.527731 21751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Bootstrap starting.
I20260806 06:41:52.529963 21751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:41:52.530200 21751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: Bootstrap complete.
I20260806 06:41:52.530429 21751 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
I20260806 06:41:52.530514 21751 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f3a86a40006a4549bb96ad7e77b671a2, State: Initialized, Role: LEARNER
I20260806 06:41:52.530550 21751 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } }
I20260806 06:41:52.530711 21751 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:52.530728 21755 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.531015 21755 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:41:52.532094 21767 catalog_manager.cc:1259] Loaded cluster ID: 48db753e8f4644bf9fdfbd2f9cb1b101
I20260806 06:41:52.532187 21767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: loading cluster ID for follower catalog manager: success
I20260806 06:41:52.533484 21767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: acquiring CA information for follower catalog manager: success
I20260806 06:41:52.533893 21767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 06:41:52.556465 21667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:52.556722 21667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:52.556747 21667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:52.557926 21667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:52.557965 21667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:52.557992 21667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:52.559680 21667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:43833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=38483
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:52.559918 21667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:52.560158 21667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.565186 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.565236 21772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.565436 21667 server_base.cc:1034] running on GCE node
W20260806 06:41:52.565285 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.565713 21667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.565968 21667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.567175 21667 hybrid_clock.cc:648] HybridClock initialized: now 1785998512567160 us; error 30 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.570199 21667 init.cc:377] Logged in from keytab as oryx/127.14.51.65@KRBTEST.COM (short username oryx)
I20260806 06:41:52.570621 21667 webserver.cc:460] Webserver started at http://127.14.51.65:38483/ using document root <none> and password file <none>
I20260806 06:41:52.570851 21667 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.570937 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.572391 21667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:52.573151 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.573444 21667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:52.573541 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "53dd49b30028417f8c5d52c954470f2e"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "b5233d0ed5253d741cc155bf5f0261df"
server_key_iv: "606e80ed5baa61c34d73d08192283e61"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.573935 21667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.591930 21667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.593672 21667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:52.593837 21667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.594089 21667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:52.594391 21667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:52.594446 21667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.594478 21667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:52.594521 21667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.603734 21667 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:43833
I20260806 06:41:52.603782 21896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:43833 every 8 connection(s)
I20260806 06:41:52.604130 21667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:52.612039 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21667
I20260806 06:41:52.612206 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21080
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:52.618046 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.605143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:38507 (local address 127.14.51.126:43891)
0806 06:41:52.605195 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.605197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.605806 (+ 609us) server_negotiation.cc:407] Connection header received
0806 06:41:52.606431 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.606433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.606448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.606499 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.607484 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608274 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.609877 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610175 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610242 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.613991 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.613993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.613993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.614000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.615860 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.616720 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.616721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.616722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.616834 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.617140 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.617141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.617141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.617339 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.617406 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.617850 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.617934 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:41:52.618510 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.605580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:38155 (local address 127.14.51.125:41275)
0806 06:41:52.605687 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.605689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.605980 (+ 291us) server_negotiation.cc:407] Connection header received
0806 06:41:52.606475 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.606477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.606490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.606522 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.607591 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608311 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.609882 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610087 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610121 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.615505 (+ 5384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.615506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.615507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.615514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.617205 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.617621 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.617622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.617623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.617681 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.617870 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.617871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.617871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.618015 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.618067 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.618289 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.618374 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:41:52.618853 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.604998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:38507)
0806 06:41:52.605460 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.605505 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.606316 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.606645 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.606649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.607033 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.607382 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.607387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608511 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.608512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.609732 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.609735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610256 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.610332 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.611303 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.611308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.613891 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.616338 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.616345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.616347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.616650 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.616913 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.616913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.616914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.617061 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.617439 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.617446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.617618 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.618149 (+ 531us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.618415 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":82,"thread_start_us":126,"threads_started":1}
I20260806 06:41:52.618983 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.605506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.14.51.65:38155)
0806 06:41:52.605786 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.605797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.606327 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.607079 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.607080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.607330 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.607502 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.607504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608511 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.608512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.609757 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.609758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610192 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.610332 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.611184 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.611189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.615402 (+ 4213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.617281 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.617282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.617283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.617526 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.617750 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.617750 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.617751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.617808 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.618114 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.618115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.618156 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.618630 (+ 474us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.618748 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":28,"thread_start_us":171,"threads_started":1}
I20260806 06:41:52.619488 21899 heartbeater.cc:347] Connected to a master server at 127.14.51.125:41275
I20260806 06:41:52.619561 21899 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.619745 21899 heartbeater.cc:511] Master 127.14.51.125:41275 requested a full tablet report, sending...
I20260806 06:41:52.619860 21898 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:52.619908 21898 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.619987 21898 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:52.620229 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.604993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:59797 (local address 127.14.51.124:33857)
0806 06:41:52.605123 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.605125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.605849 (+ 724us) server_negotiation.cc:407] Connection header received
0806 06:41:52.606495 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.606496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.606511 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.606786 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.607539 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608332 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.609729 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610059 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610122 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.615194 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.615196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.615196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.615204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.618040 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.618602 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.618603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.618603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.618665 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.618918 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.618919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.618920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.619076 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.619144 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.620056 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.620144 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260806 06:41:52.620244 20878 ts_manager.cc:194] Re-registered known tserver with Master: 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
I20260806 06:41:52.620254 21513 ts_manager.cc:194] Registered new tserver with Master: 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
I20260806 06:41:52.620486 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.604998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.14.51.65:59797)
0806 06:41:52.605460 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.605505 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.606316 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.606817 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.606818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.607190 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.607383 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.607397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.608513 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.608514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.609634 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.609636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.610189 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.610190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.610332 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.610885 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.610895 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.614782 (+ 3887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.618178 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.618178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.618179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.618515 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.618744 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.618745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.618745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.618827 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.619191 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.619192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.619233 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.619297 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.619405 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":79,"thread_start_us":125,"threads_started":1}
I20260806 06:41:52.620743 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:38507
I20260806 06:41:52.620826 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:42405
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:43891
--webserver_port=38655
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:41:52.621238 21897 heartbeater.cc:347] Connected to a master server at 127.14.51.124:33857
I20260806 06:41:52.621291 21897 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.621493 21897 heartbeater.cc:511] Master 127.14.51.124:33857 requested a full tablet report, sending...
I20260806 06:41:52.621835 21705 ts_manager.cc:194] Registered new tserver with Master: 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
W20260806 06:41:52.701702 21905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:52.701913 21905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:52.701936 21905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:52.703208 21905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:52.703248 21905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:52.703275 21905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:52.704721 21905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:42405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=38655
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:52.704888 21905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:52.705045 21905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.710219 21911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.710217 21905 server_base.cc:1034] running on GCE node
W20260806 06:41:52.710187 21910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.710182 21913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.710702 21905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.710973 21905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.712133 21905 hybrid_clock.cc:648] HybridClock initialized: now 1785998512712126 us; error 23 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.714641 21905 init.cc:377] Logged in from keytab as oryx/127.14.51.66@KRBTEST.COM (short username oryx)
I20260806 06:41:52.714993 21905 webserver.cc:460] Webserver started at http://127.14.51.66:38655/ using document root <none> and password file <none>
I20260806 06:41:52.715224 21905 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.715332 21905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.716681 21905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:52.717257 21920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.717408 21905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:52.717489 21905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "d2b840b3392e4f8cac58383a1d003743"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "50ee099518e3a93cb969b7134d0c7dfc"
server_key_iv: "de7ad180d466c70ffc3fff6ac7cd811f"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.717779 21905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.728139 21905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.729789 21905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:52.729893 21905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.730050 21905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:52.730274 21905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:52.730304 21905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.730324 21905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:52.730335 21905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.736123 21905 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:42405
I20260806 06:41:52.736194 22035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:42405 every 8 connection(s)
I20260806 06:41:52.736526 21905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:52.745982 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 21905
I20260806 06:41:52.746143 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21223
I20260806 06:41:52.749010 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:34421 (local address 127.14.51.124:33857)
0806 06:41:52.737704 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.737707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.737718 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:41:52.738726 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.738728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.738746 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.738796 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.739696 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740206 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741181 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.741398 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741475 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.743945 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.743947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.743948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.743955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.745653 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.747860 (+ 2207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.747862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.747862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.747940 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748227 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.748228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.748229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.748368 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.748422 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.748771 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.748874 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 06:41:52.749574 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:37765 (local address 127.14.51.126:43891)
0806 06:41:52.737260 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.737262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.737674 (+ 412us) server_negotiation.cc:407] Connection header received
0806 06:41:52.738731 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.738733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.738745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.738779 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.739711 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740475 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741677 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.741937 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.742033 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.745426 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.745427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.745428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.745433 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.748371 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748752 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.748753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.748753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.748811 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.749010 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.749011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.749011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.749144 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.749194 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.749393 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.749490 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:41:52.749965 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:37765)
0806 06:41:52.737375 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.737416 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.738637 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.738860 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.738862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.739417 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.739600 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.739606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740574 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.740574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.741596 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.741597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.742113 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.742114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.742173 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.742763 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.742768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.745162 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.748460 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.748461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.748662 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.748887 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748887 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.748888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.748942 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.749244 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.749245 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.749282 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.749696 (+ 414us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.749783 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":24,"thread_start_us":91,"threads_started":1}
I20260806 06:41:52.750015 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:43403 (local address 127.14.51.125:41275)
0806 06:41:52.737719 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.737721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.737839 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:41:52.738731 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.738732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.738745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.738779 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.739711 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740473 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741637 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.741908 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741951 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.745001 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.745002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.745002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.745010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.746705 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748968 (+ 2263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.748970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.748970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.749040 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.749308 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.749309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.749309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.749474 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.749539 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.749780 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.749875 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:41:52.750521 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.14.51.66:43403)
0806 06:41:52.737735 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.737745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.738629 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.738864 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.738865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.739397 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.739600 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.739606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740574 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.740574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.741521 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.741523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.742042 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.742043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.742107 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.742665 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.742670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.744914 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.748581 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.748583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.748859 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.749141 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.749142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.749142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.749218 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.749579 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.749580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.749630 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.750149 (+ 519us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.750276 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":95,"thread_start_us":70,"threads_started":1}
I20260806 06:41:52.750727 22037 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:52.750818 22037 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.750975 22038 heartbeater.cc:347] Connected to a master server at 127.14.51.125:41275
I20260806 06:41:52.751051 22038 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.751114 22038 heartbeater.cc:511] Master 127.14.51.125:41275 requested a full tablet report, sending...
I20260806 06:41:52.750993 22037 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:52.749965 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.737220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.14.51.66:34421)
0806 06:41:52.737585 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.737596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.738609 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.738858 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.738862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.739367 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.739598 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.739603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.740315 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.740316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.741086 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.741088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.741482 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.741483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.741604 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.742153 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.742162 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.743849 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.745727 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.745733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.745735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.746126 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.748019 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.748020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.748020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.748147 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.748459 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.748465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.748614 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.749355 (+ 741us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.749514 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":25,"thread_start_us":94,"threads_started":1}
I20260806 06:41:52.751619 22036 heartbeater.cc:347] Connected to a master server at 127.14.51.124:33857
I20260806 06:41:52.751660 22036 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.751717 22036 heartbeater.cc:511] Master 127.14.51.124:33857 requested a full tablet report, sending...
I20260806 06:41:52.751981 21705 ts_manager.cc:194] Registered new tserver with Master: d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:52.752247 21513 ts_manager.cc:194] Registered new tserver with Master: d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:52.755739 20878 ts_manager.cc:194] Re-registered known tserver with Master: d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:52.755877 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:37333
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:43891
--webserver_port=42095
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:41:52.756222 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:37765
W20260806 06:41:52.837730 22044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:52.837954 22044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:52.837996 22044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:52.839112 22044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:52.839206 22044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:52.839247 22044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:52.840807 22044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:37333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=42095
--tserver_master_addrs=127.14.51.126:43891,127.14.51.125:41275,127.14.51.124:33857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:52.841073 22044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:52.841295 22044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:52.846789 22052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.846789 22050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:52.846798 22049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:52.846984 22044 server_base.cc:1034] running on GCE node
I20260806 06:41:52.847234 22044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:52.847432 22044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:52.848592 22044 hybrid_clock.cc:648] HybridClock initialized: now 1785998512848577 us; error 27 us; skew 500 ppm
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:52.850840 22044 init.cc:377] Logged in from keytab as oryx/127.14.51.67@KRBTEST.COM (short username oryx)
I20260806 06:41:52.851301 22044 webserver.cc:460] Webserver started at http://127.14.51.67:42095/ using document root <none> and password file <none>
I20260806 06:41:52.851504 22044 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:52.851589 22044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:52.852792 22044 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:52.853281 22059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.853435 22044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:52.853525 22044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "09a149d9783b4f26b1282f2f38a8ac6a"
format_stamp: "Formatted at 2026-08-06 06:41:51 on dist-test-slave-1xrh"
server_key: "7a0477738f71046c0ad78e833aefc4bb"
server_key_iv: "3c08b756934229337e103e0f35b329bc"
server_key_version: "encryptionkey@0"
I20260806 06:41:52.853845 22044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:52.864691 22044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:52.866318 22044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:52.866484 22044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:52.866694 22044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:41:52.867005 22044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:52.867064 22044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.867105 22044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:52.867156 22044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:52.873618 22044 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37333
I20260806 06:41:52.873661 22175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37333 every 8 connection(s)
I20260806 06:41:52.874025 22044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.125@KRBTEST.COM
I20260806 06:41:52.880981 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 22044
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
Aug 06 06:41:52 dist-test-slave-1xrh krb5kdc[20832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998512, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.124@KRBTEST.COM
I20260806 06:41:52.885468 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.874942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.14.51.67:52309)
0806 06:41:52.875318 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.875327 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.875743 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.876144 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.876145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.876453 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.876696 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.876698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.877416 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.877417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.878663 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.878666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879201 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.879322 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.879824 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.879833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.881444 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.883542 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.883548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.883550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.883861 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.884140 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.884141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.884141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.884315 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.884703 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.884709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.884921 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.885036 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.885194 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":64,"threads_started":1}
I20260806 06:41:52.885942 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.875102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:52309 (local address 127.14.51.125:41275)
0806 06:41:52.875180 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.875182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.875492 (+ 310us) server_negotiation.cc:407] Connection header received
0806 06:41:52.876069 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.876070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.876079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.876135 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.876820 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.877300 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.878749 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879053 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879185 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.881580 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.881582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.881582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.881589 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.883436 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.883969 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.883970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.883971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.884036 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.884372 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.884373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.884373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.884591 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.884675 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.885739 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.885820 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 06:41:52.886597 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.874924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:47687 (local address 127.14.51.126:43891)
0806 06:41:52.874970 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.874972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.875233 (+ 261us) server_negotiation.cc:407] Connection header received
0806 06:41:52.875981 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.875982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.875991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.876029 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.876769 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.877311 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879079 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879369 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879457 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.883155 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.883157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.883157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.883164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.885296 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.885704 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.885705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.885705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.885763 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.885969 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.885969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.885970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.886090 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.886147 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.886432 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.886504 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:52.886799 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.874650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:47687)
0806 06:41:52.875046 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.875054 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.875732 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.876051 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.876052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.876445 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.876675 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.876680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.878343 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.878344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.878986 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.878988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879465 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.879560 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.880168 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.880173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.883060 (+ 2887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.885378 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.885378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.885379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.885606 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.885834 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.885834 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.885835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.885894 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.886176 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.886177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.886221 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.886286 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.886686 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":5,"spinlock_wait_cycles":8064,"thread_start_us":88,"threads_started":1}
I20260806 06:41:52.886871 22178 heartbeater.cc:347] Connected to a master server at 127.14.51.125:41275
I20260806 06:41:52.886982 22178 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.887171 22178 heartbeater.cc:511] Master 127.14.51.125:41275 requested a full tablet report, sending...
I20260806 06:41:52.887346 22177 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43891
I20260806 06:41:52.887405 22177 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.887470 22177 heartbeater.cc:511] Master 127.14.51.126:43891 requested a full tablet report, sending...
I20260806 06:41:52.887578 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.874646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:59447 (local address 127.14.51.124:33857)
0806 06:41:52.874716 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.874719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.875171 (+ 452us) server_negotiation.cc:407] Connection header received
0806 06:41:52.875832 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.875834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.875847 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.875897 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.877474 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.878243 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879187 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879410 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879470 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.883556 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.883557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.883557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.883563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.886034 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.886628 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.886629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.886629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.886687 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.886879 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.886879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.886880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.887030 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.887094 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.887420 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.887498 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:41:52.887578 21513 ts_manager.cc:194] Registered new tserver with Master: 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333)
I20260806 06:41:52.887696 20878 ts_manager.cc:194] Re-registered known tserver with Master: 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333)
I20260806 06:41:52.888180 20878 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:47687
I20260806 06:41:52.889201 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.874649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.14.51.67:59447)
0806 06:41:52.874980 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.875018 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.875732 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.875983 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.875985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.876544 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:52.876675 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.876680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.878317 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.878318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.879076 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.879079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.879542 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.879543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.879599 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.880081 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:52.880085 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:52.882786 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:52.886296 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.886296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.886297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.886551 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.886752 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:52.886752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:52.886753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:52.886807 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:52.887123 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:52.887124 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:52.887182 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.887249 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.887341 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":68,"thread_start_us":84,"threads_started":1}
I20260806 06:41:52.889529 22176 heartbeater.cc:347] Connected to a master server at 127.14.51.124:33857
I20260806 06:41:52.889580 22176 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:52.889643 22176 heartbeater.cc:511] Master 127.14.51.124:33857 requested a full tablet report, sending...
I20260806 06:41:52.889889 21705 ts_manager.cc:194] Registered new tserver with Master: 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333)
I20260806 06:41:52.893501 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.883190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.14.51.124:33857)
0806 06:41:52.883296 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.883298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.884593 (+ 1295us) server_negotiation.cc:407] Connection header received
0806 06:41:52.884730 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.884731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.884742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.884909 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.885947 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.886613 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.888631 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.888850 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.888922 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.890616 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.890617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.890617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.890623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.892378 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.892744 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.892745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.892746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.892808 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.893015 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.893015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.893016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.893144 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.893250 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.893343 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.893413 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":69,"threads_started":1}
I20260806 06:41:52.895023 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.883504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.14.51.125:41275)
0806 06:41:52.884060 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.886348 (+ 2288us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.886356 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:52.886376 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.886377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.886388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.886445 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.887624 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.888328 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.889601 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.889884 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.889935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.891305 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.891306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.891306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.891313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.893714 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.894193 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.894194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.894194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.894271 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.894465 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.894466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.894467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.894588 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.894689 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.894763 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.894931 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":145,"threads_started":1}
I20260806 06:41:52.895947 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.883068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.14.51.126:43891)
0806 06:41:52.884014 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.884016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.884023 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:52.884042 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.884043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.884055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.884304 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:52.886483 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.889927 (+ 3444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.890996 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.891345 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.891432 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.892824 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:52.892826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:52.892826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:52.892832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:52.894870 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.895225 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.895227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.895227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.895291 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:52.895502 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:52.895503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:52.895503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:52.895600 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:52.895670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.895774 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.895848 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":435,"threads_started":1}
I20260806 06:41:52.897322 20878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:52.898833 20878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:52.906492 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.902636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43833 (local address 127.0.0.1:44344)
0806 06:41:52.902793 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.902810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.902995 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.903492 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.903493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.903496 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:52.903536 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.903537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.904393 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.904394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.905214 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.905216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.906256 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.906257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.906321 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.906326 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.906377 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.906413 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260806 06:41:52.906821 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.902856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.14.51.65:43833)
0806 06:41:52.903117 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.903121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.903134 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:41:52.903199 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.903201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.903217 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.903438 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:52.903590 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.904301 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.905387 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.905934 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.906030 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.906216 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.906644 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.906691 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":126,"threads_started":1}
I20260806 06:41:52.907171 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.902568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37333 (local address 127.0.0.1:51678)
0806 06:41:52.902599 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.902610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.902649 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.903637 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.903638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.903640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:52.903682 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.903683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.904649 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.904650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.905756 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.905759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.906944 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.906946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.907005 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.907008 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.907038 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.907091 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":4}
I20260806 06:41:52.907673 21831 tablet_service.cc:1467] Processing CreateTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 (DEFAULT_TABLE table=test-table [id=923ca9bf6d1a46f5b8e8f5233afd0a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:52.908007 21831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc5e4fc58d54a6782ed41462fba00c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:52.909519 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.902672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42405 (local address 127.0.0.1:40688)
0806 06:41:52.903658 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.903669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.903876 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.904890 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.904891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.904893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:52.904927 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.904928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.907109 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.907110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.908232 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.908236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.909275 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.909276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.909332 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.909336 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.909408 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.909445 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":335,"threads_started":1}
I20260806 06:41:52.909579 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.903204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.14.51.66:42405)
0806 06:41:52.904731 (+ 1527us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.904734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.904743 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:52.904778 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.904780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.904795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.904872 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:52.904992 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.906466 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.908336 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.909078 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.909150 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.909236 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.909431 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.909480 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":234,"threads_started":1}
I20260806 06:41:52.910503 21970 tablet_service.cc:1467] Processing CreateTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 (DEFAULT_TABLE table=test-table [id=923ca9bf6d1a46f5b8e8f5233afd0a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:52.911003 21970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc5e4fc58d54a6782ed41462fba00c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:52.912159 22203 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Bootstrap starting.
I20260806 06:41:52.912710 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.902660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.14.51.67:37333)
0806 06:41:52.903106 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.903111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.903121 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:52.903171 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.903173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.903188 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.903878 (+ 690us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:52.903928 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.904562 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.905846 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.906831 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.912151 (+ 5320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.912277 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.912307 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.912350 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":79,"threads_started":1}
I20260806 06:41:52.913511 22203 tablet_bootstrap.cc:654] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:52.914535 22203 log.cc:824] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:52.916040 22203 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: No bootstrap required, opened a new log
I20260806 06:41:52.916055 22205 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Bootstrap starting.
I20260806 06:41:52.916168 22203 ts_tablet_manager.cc:1397] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 06:41:52.916653 22108 tablet_service.cc:1467] Processing CreateTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 (DEFAULT_TABLE table=test-table [id=923ca9bf6d1a46f5b8e8f5233afd0a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:52.917039 22205 tablet_bootstrap.cc:654] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:52.917143 22108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc5e4fc58d54a6782ed41462fba00c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:52.917374 22205 log.cc:824] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:52.919885 22203 raft_consensus.cc:348] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.920307 22203 raft_consensus.cc:374] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:52.920358 22203 raft_consensus.cc:729] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53dd49b30028417f8c5d52c954470f2e, State: Initialized, Role: FOLLOWER
I20260806 06:41:52.920511 22203 consensus_queue.cc:260] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.920819 22203 ts_tablet_manager.cc:1428] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 06:41:52.921168 21898 heartbeater.cc:503] Master 127.14.51.126:43891 was elected leader, sending a full tablet report...
I20260806 06:41:52.922037 22208 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Bootstrap starting.
I20260806 06:41:52.922740 22208 tablet_bootstrap.cc:654] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:52.923005 22208 log.cc:824] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:52.923786 22205 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: No bootstrap required, opened a new log
I20260806 06:41:52.923828 22208 tablet_bootstrap.cc:492] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: No bootstrap required, opened a new log
I20260806 06:41:52.923887 22205 ts_tablet_manager.cc:1397] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 06:41:52.923872 22208 ts_tablet_manager.cc:1397] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:41:52.925683 22208 raft_consensus.cc:348] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.925794 22208 raft_consensus.cc:374] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:52.925813 22208 raft_consensus.cc:729] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a149d9783b4f26b1282f2f38a8ac6a, State: Initialized, Role: FOLLOWER
I20260806 06:41:52.925926 22208 consensus_queue.cc:260] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.926218 22205 raft_consensus.cc:348] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.926347 22208 ts_tablet_manager.cc:1428] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:41:52.926400 22205 raft_consensus.cc:374] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:52.926440 22177 heartbeater.cc:503] Master 127.14.51.126:43891 was elected leader, sending a full tablet report...
I20260806 06:41:52.926450 22205 raft_consensus.cc:729] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b840b3392e4f8cac58383a1d003743, State: Initialized, Role: FOLLOWER
I20260806 06:41:52.926608 22205 consensus_queue.cc:260] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:52.926900 22205 ts_tablet_manager.cc:1428] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:52.927043 22037 heartbeater.cc:503] Master 127.14.51.126:43891 was elected leader, sending a full tablet report...
I20260806 06:41:52.958847 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.956475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.14.51.124:33857)
0806 06:41:52.956521 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:52.956522 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:52.956531 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:52.956595 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:52.956595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:52.956614 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:52.956668 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:52.956822 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.957323 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.958218 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.958602 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.958660 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.958705 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:52.958768 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:41:52.958793 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:52.958850 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:52.956387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.0.0.1:44094)
0806 06:41:52.956455 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:52.956469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:52.956521 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:52.956692 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:52.956693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:52.956694 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:52.956735 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.956737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.957391 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.957392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.958142 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:52.958145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:52.958679 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:52.958680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:52.958723 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:52.958726 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:52.958765 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:41:52.958798 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:41:52.959227 21723 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 06:41:52.959630 21755 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0c4627528642fcaf93f3f4a7f002c3. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:52.959780 21755 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: LEARNER
W20260806 06:41:52.987340 22040 tablet.cc:2369] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:53.105345 21900 tablet.cc:2369] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:53.124944 22179 tablet.cc:2369] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:53.187045 22210 raft_consensus.cc:482] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:53.187216 22210 raft_consensus.cc:504] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:53.187664 22210 leader_election.cc:288] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833), d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:53.190815 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.187957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:37369 (local address 127.14.51.65:43833)
0806 06:41:53.188014 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.188016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.188025 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:53.188067 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.188067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.188074 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.188146 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:53.188309 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.188805 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.189963 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.190463 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.190524 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.190532 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.190712 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.190749 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:41:53.190815 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.187946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:33921 (local address 127.14.51.66:42405)
0806 06:41:53.188015 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.188017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.188026 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:53.188060 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.188061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.188068 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.188113 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:53.188351 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.188815 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.189911 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.190437 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.190516 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.190525 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.190712 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.190747 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:41:53.190819 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.187839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43833 (local address 127.14.51.67:37369)
0806 06:41:53.187908 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:53.187919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:53.187965 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:53.188167 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:53.188168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:53.188169 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:53.188233 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:53.188235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.188887 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.188888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:53.189827 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:53.189829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.190536 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.190537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:53.190628 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.190632 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:53.190689 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:41:53.190728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":7}
I20260806 06:41:53.190830 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.187867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42405 (local address 127.14.51.67:33921)
0806 06:41:53.187909 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:53.187921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:53.187970 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:53.188136 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:53.188136 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:53.188138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:53.188252 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:53.188254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.188933 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.188934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:53.189822 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:53.189825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.190536 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.190537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:53.190629 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.190632 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:53.190689 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:41:53.190728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":83}
I20260806 06:41:53.191169 21851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc5e4fc58d54a6782ed41462fba00c8" candidate_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53dd49b30028417f8c5d52c954470f2e" is_pre_election: true
I20260806 06:41:53.191310 21851 raft_consensus.cc:2454] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a149d9783b4f26b1282f2f38a8ac6a in term 0.
I20260806 06:41:53.191578 22060 leader_election.cc:302] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a149d9783b4f26b1282f2f38a8ac6a, 53dd49b30028417f8c5d52c954470f2e; no voters:
I20260806 06:41:53.191582 21990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc5e4fc58d54a6782ed41462fba00c8" candidate_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b840b3392e4f8cac58383a1d003743" is_pre_election: true
I20260806 06:41:53.191712 21990 raft_consensus.cc:2454] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a149d9783b4f26b1282f2f38a8ac6a in term 0.
I20260806 06:41:53.191728 22210 raft_consensus.cc:2781] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:53.191823 22210 raft_consensus.cc:482] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:53.191857 22210 raft_consensus.cc:3037] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:53.192598 22210 raft_consensus.cc:504] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:53.192776 22210 leader_election.cc:288] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [CANDIDATE]: Term 1 election: Requested vote from peers 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833), d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405)
I20260806 06:41:53.192919 21990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc5e4fc58d54a6782ed41462fba00c8" candidate_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b840b3392e4f8cac58383a1d003743"
I20260806 06:41:53.192943 21851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc5e4fc58d54a6782ed41462fba00c8" candidate_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53dd49b30028417f8c5d52c954470f2e"
I20260806 06:41:53.193012 21851 raft_consensus.cc:3037] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:53.193018 21990 raft_consensus.cc:3037] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:53.193778 21990 raft_consensus.cc:2454] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a149d9783b4f26b1282f2f38a8ac6a in term 1.
I20260806 06:41:53.193995 22063 leader_election.cc:302] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a149d9783b4f26b1282f2f38a8ac6a, d2b840b3392e4f8cac58383a1d003743; no voters:
I20260806 06:41:53.194089 22210 raft_consensus.cc:2781] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:53.194228 22210 raft_consensus.cc:686] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 LEADER]: Becoming Leader. State: Replica: 09a149d9783b4f26b1282f2f38a8ac6a, State: Running, Role: LEADER
I20260806 06:41:53.194252 21851 raft_consensus.cc:2454] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a149d9783b4f26b1282f2f38a8ac6a in term 1.
I20260806 06:41:53.194334 22210 consensus_queue.cc:237] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } }
I20260806 06:41:53.195119 20878 catalog_manager.cc:5495] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67). New cstate: current_term: 1 leader_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:53.195787 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: NON_VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: true } } }
I20260806 06:41:53.195888 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:53.196648 21471 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: attempting to promote NON_VOTER f3a86a40006a4549bb96ad7e77b671a2 to VOTER
I20260806 06:41:53.196753 21471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:53.197443 21723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 LEARNER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:41:53.197454 21523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Refusing update from remote peer 9c0c4627528642fcaf93f3f4a7f002c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:41:53.197664 21471 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:41:53.197767 22217 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:41:53.198474 21471 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.198638 21523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.198627 21723 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199502 22219 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199548 21471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199596 22219 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:53.199617 21471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:53.199685 22217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199695 21471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199786 21471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:53.199777 22217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:53.199887 21755 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:53.199970 21755 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:53.263628 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.260894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.14.51.67:37333)
0806 06:41:53.260949 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.260952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.260959 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:53.261030 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.261031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.261048 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.261110 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:53.261201 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.262007 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.262890 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.263079 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.263148 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.263299 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:53.263393 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:53.263453 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.263487 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.263505 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:41:53.265580 21990 raft_consensus.cc:1261] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Refusing update from remote peer 09a149d9783b4f26b1282f2f38a8ac6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:53.265686 21851 raft_consensus.cc:1261] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Refusing update from remote peer 09a149d9783b4f26b1282f2f38a8ac6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:53.266008 22215 consensus_queue.cc:1035] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:53.266145 22210 consensus_queue.cc:1035] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:53.269117 22221 mvcc.cc:205] Tried to move back new op lower bound from 7315449910333059072 to 7315449910044524544. Current Snapshot: MvccSnapshot[applied={T|T < 7315449910333059072 or (T in {7315449910333059072})}]
I20260806 06:41:53.271956 20878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:53.272047 20878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:53.273375 20878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c0c4627528642fcaf93f3f4a7f002c3: Sending DeleteTablet for 3 replicas of tablet ffc5e4fc58d54a6782ed41462fba00c8
I20260806 06:41:53.273697 21831 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:53 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:44344
I20260806 06:41:53.273706 22108 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:53 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:51678
I20260806 06:41:53.274082 21970 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc5e4fc58d54a6782ed41462fba00c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:53 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:40688
I20260806 06:41:53.275063 22233 tablet_replica.cc:330] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: stopping tablet replica
I20260806 06:41:53.275161 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 20845
I20260806 06:41:53.276520 22233 raft_consensus.cc:2229] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:53.276638 22233 raft_consensus.cc:2258] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:53.277333 22233 ts_tablet_manager.cc:1905] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:53.279407 22233 ts_tablet_manager.cc:1918] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:53.279484 22233 log.cc:1196] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/ffc5e4fc58d54a6782ed41462fba00c8
I20260806 06:41:53.279764 22233 ts_tablet_manager.cc:1939] T ffc5e4fc58d54a6782ed41462fba00c8 P d2b840b3392e4f8cac58383a1d003743: Deleting consensus metadata
W20260806 06:41:53.282006 21924 connection.cc:535] server connection from 127.0.0.1:40688 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:41:53.283216 22231 tablet_replica.cc:330] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: stopping tablet replica
I20260806 06:41:53.283237 22232 tablet_replica.cc:330] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: stopping tablet replica
I20260806 06:41:53.283327 22232 raft_consensus.cc:2229] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:53.283327 22231 raft_consensus.cc:2229] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:53.283416 22232 raft_consensus.cc:2258] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:53.283432 22231 raft_consensus.cc:2258] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:53.283921 22231 ts_tablet_manager.cc:1905] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:53.285270 22232 ts_tablet_manager.cc:1905] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:53.285737 22231 ts_tablet_manager.cc:1918] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:53.285820 22231 log.cc:1196] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/ffc5e4fc58d54a6782ed41462fba00c8
I20260806 06:41:53.286134 22231 ts_tablet_manager.cc:1939] T ffc5e4fc58d54a6782ed41462fba00c8 P 53dd49b30028417f8c5d52c954470f2e: Deleting consensus metadata
I20260806 06:41:53.287312 22232 ts_tablet_manager.cc:1918] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:53.287388 22232 log.cc:1196] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/ffc5e4fc58d54a6782ed41462fba00c8
I20260806 06:41:53.287650 22232 ts_tablet_manager.cc:1939] T ffc5e4fc58d54a6782ed41462fba00c8 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting consensus metadata
W20260806 06:41:53.287990 21785 connection.cc:441] server connection from 127.0.0.1:44344 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44344 (request call id 1) could send its response
W20260806 06:41:53.288086 22063 connection.cc:441] server connection from 127.0.0.1:51678 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51678 (request call id 1) could send its response
I20260806 06:41:53.288159 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.274546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.14.51.124:33857)
0806 06:41:53.274586 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.274588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.274740 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:41:53.274832 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.274833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.274843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.274870 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:53.280485 (+ 5615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.281296 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.282513 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.282838 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.282881 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.284906 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:53.284907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:53.284908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:53.284915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:53.286878 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.287325 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.287326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.287326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.287389 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.287582 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.287583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.287583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.287703 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:53.287787 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.287975 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.288065 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:53.292955 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.284837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.14.51.125:41275)
0806 06:41:53.284879 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.284881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.284889 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:53.284922 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.284923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.284936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.285017 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:53.285710 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.286490 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.287867 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.288167 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.288214 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.290039 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:53.290048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:53.290049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:53.290056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:53.291740 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.292133 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.292135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.292135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.292193 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.292428 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.292429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.292429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.292553 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:53.292660 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.292759 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.292848 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:53.295554 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:53.284753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.14.51.124:33857)
0806 06:41:53.284793 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:53.284795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:53.288650 (+ 3855us) server_negotiation.cc:407] Connection header received
0806 06:41:53.288708 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:53.288708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:53.288719 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:53.288775 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:53.289471 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.290165 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.291225 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:53.291519 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:53.291607 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:53.293101 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:53.293102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:53.293102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:53.293109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:53.294561 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.294867 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.294868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.294868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.294931 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:53.295092 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:53.295092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:53.295092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:53.295229 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:53.295323 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:53.295390 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:41:53.295463 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:41:54.270562 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.270296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:57807)
0806 06:41:54.270448 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.270517 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
W20260806 06:41:54.270857 22037 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43891 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.271996 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.271721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:60471)
0806 06:41:54.271882 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.271952 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":50,"threads_started":1}
I20260806 06:41:54.272157 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.271844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:57955)
0806 06:41:54.272044 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.272115 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":55,"threads_started":1}
W20260806 06:41:54.272315 21898 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43891 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
W20260806 06:41:54.272536 22177 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43891 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.773931 22245 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c0c4627528642fcaf93f3f4a7f002c3)
I20260806 06:41:54.774034 22246 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c0c4627528642fcaf93f3f4a7f002c3)
I20260806 06:41:54.774073 22245 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:54.774123 22246 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:54.774338 22246 leader_election.cc:288] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857)
I20260806 06:41:54.774376 22245 leader_election.cc:288] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125:41275)
I20260806 06:41:54.774785 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45424)
0806 06:41:54.774677 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.774735 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":49,"threads_started":1}
W20260806 06:41:54.774955 21496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.775019 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45434)
0806 06:41:54.774915 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.774971 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":74,"threads_started":1}
W20260806 06:41:54.775147 21698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.775604 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.775507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45456)
0806 06:41:54.775556 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.775578 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:41:54.775602 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.775495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45440)
0806 06:41:54.775546 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.775575 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:41:54.775754 21698 leader_election.cc:334] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
W20260806 06:41:54.775873 21496 leader_election.cc:334] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.778270 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.14.51.125:41275)
0806 06:41:54.774948 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:54.774950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:54.774958 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:54.774978 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:54.774979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:54.774987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:54.775269 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:54.775399 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.776120 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.777518 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.777960 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.778016 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:54.778025 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:54.778158 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:41:54.778205 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":46,"threads_started":1}
I20260806 06:41:54.778275 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41275 (local address 127.0.0.1:33150)
0806 06:41:54.774646 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.774658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:54.774716 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:54.775247 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:54.775248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:54.775250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:54.775318 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:54.775320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.776194 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.776195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:54.777437 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:54.777440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.778032 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.778033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:54.778090 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:54.778094 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:54.778146 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:41:54.778205 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":24,"thread_start_us":54,"threads_started":1}
I20260806 06:41:54.778352 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:33857 (local address 127.0.0.1:44110)
0806 06:41:54.774740 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.774751 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:54.774825 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:54.775078 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:54.775079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:54.775081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:54.775162 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:54.775164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.776192 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.776193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:54.777437 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:54.777440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.778086 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.778087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:54.778148 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:54.778152 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:54.778209 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:41:54.778246 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":47,"threads_started":1}
I20260806 06:41:54.778538 22252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.774680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.14.51.124:33857)
0806 06:41:54.774899 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:54.774901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:54.774910 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:54.774935 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:54.774935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:54.774945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:54.775017 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:54.775260 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.776081 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.777552 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:54.777983 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:54.778068 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:54.778080 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:54.778223 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:41:54.778262 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":36,"threads_started":1}
I20260806 06:41:54.778782 21523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a86a40006a4549bb96ad7e77b671a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" is_pre_election: true
I20260806 06:41:54.778781 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a86a40006a4549bb96ad7e77b671a2" is_pre_election: true
I20260806 06:41:54.778887 21523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3a86a40006a4549bb96ad7e77b671a2 in term 1.
I20260806 06:41:54.778913 21723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3dc4b71b1004aecbba94ae63fcad0fa in term 1.
I20260806 06:41:54.779076 21494 leader_election.cc:302] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3a86a40006a4549bb96ad7e77b671a2, f3dc4b71b1004aecbba94ae63fcad0fa; no voters: 9c0c4627528642fcaf93f3f4a7f002c3
I20260806 06:41:54.779083 21697 leader_election.cc:302] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3a86a40006a4549bb96ad7e77b671a2, f3dc4b71b1004aecbba94ae63fcad0fa; no voters: 9c0c4627528642fcaf93f3f4a7f002c3
I20260806 06:41:54.779204 22246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:41:54.779201 22245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:41:54.779246 22245 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c0c4627528642fcaf93f3f4a7f002c3)
I20260806 06:41:54.779273 22245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:41:54.779247 22246 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c0c4627528642fcaf93f3f4a7f002c3)
I20260806 06:41:54.779299 22246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:41:54.779969 22246 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:54.779984 22245 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:54.780112 22246 leader_election.cc:288] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 election: Requested vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857)
I20260806 06:41:54.780125 22245 leader_election.cc:288] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 election: Requested vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3dc4b71b1004aecbba94ae63fcad0fa (127.14.51.125:41275)
I20260806 06:41:54.780267 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.780189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45468)
0806 06:41:54.780220 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.780247 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 06:41:54.780299 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a86a40006a4549bb96ad7e77b671a2"
I20260806 06:41:54.780364 21723 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f3dc4b71b1004aecbba94ae63fcad0fa in current term 2: Already voted for candidate f3a86a40006a4549bb96ad7e77b671a2 in this term.
I20260806 06:41:54.780347 21523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a86a40006a4549bb96ad7e77b671a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa"
I20260806 06:41:54.780431 21523 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f3a86a40006a4549bb96ad7e77b671a2 in current term 2: Already voted for candidate f3dc4b71b1004aecbba94ae63fcad0fa in this term.
I20260806 06:41:54.780432 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.780206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45460)
0806 06:41:54.780385 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.780413 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260806 06:41:54.780766 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.780689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45486)
0806 06:41:54.780720 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.780744 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 06:41:54.780802 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:54.780660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45472)
0806 06:41:54.780706 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:54.780751 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:41:54.780874 21496 leader_election.cc:334] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.780916 21496 leader_election.cc:302] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3dc4b71b1004aecbba94ae63fcad0fa; no voters: 9c0c4627528642fcaf93f3f4a7f002c3, f3a86a40006a4549bb96ad7e77b671a2
W20260806 06:41:54.780889 21698 leader_election.cc:334] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:54.780958 21698 leader_election.cc:302] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a86a40006a4549bb96ad7e77b671a2; no voters: 9c0c4627528642fcaf93f3f4a7f002c3, f3dc4b71b1004aecbba94ae63fcad0fa
I20260806 06:41:54.781019 22245 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 06:41:54.781075 22246 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 06:41:55.271493 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:55.271259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:56459)
0806 06:41:55.271401 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:55.271443 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260806 06:41:55.272886 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:55.272586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:34205)
0806 06:41:55.272794 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:55.272830 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":65,"threads_started":1}
I20260806 06:41:55.273208 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:55.272860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:36993)
0806 06:41:55.273124 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:55.273163 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
I20260806 06:41:56.272289 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.271987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:57677)
0806 06:41:56.272164 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.272208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260806 06:41:56.273574 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.273317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:33725)
0806 06:41:56.273493 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.273528 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":55,"threads_started":1}
I20260806 06:41:56.273854 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.273605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:35813)
0806 06:41:56.273748 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.273775 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":49,"threads_started":1}
I20260806 06:41:56.642872 22261 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:56.642974 22261 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:56.643179 22261 leader_election.cc:288] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857)
I20260806 06:41:56.643436 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a86a40006a4549bb96ad7e77b671a2" is_pre_election: true
I20260806 06:41:56.643524 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.643303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45726)
0806 06:41:56.643436 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.643482 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260806 06:41:56.643554 21723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3dc4b71b1004aecbba94ae63fcad0fa in term 2.
I20260806 06:41:56.643723 21494 leader_election.cc:302] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3a86a40006a4549bb96ad7e77b671a2, f3dc4b71b1004aecbba94ae63fcad0fa; no voters:
I20260806 06:41:56.643846 22261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 06:41:56.643930 22261 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:56.643955 22261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:41:56.644006 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.643862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45734)
0806 06:41:56.643919 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.643953 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:41:56.644080 21496 leader_election.cc:334] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:56.644760 22261 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:56.644912 22261 leader_election.cc:288] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 election: Requested vote from peers 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891), f3a86a40006a4549bb96ad7e77b671a2 (127.14.51.124:33857)
I20260806 06:41:56.645198 21723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a86a40006a4549bb96ad7e77b671a2"
I20260806 06:41:56.645226 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.645115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45736)
0806 06:41:56.645172 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.645201 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:41:56.645268 21723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:41:56.645579 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.645494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45748)
0806 06:41:56.645543 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.645565 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:41:56.645674 21496 leader_election.cc:334] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
I20260806 06:41:56.646090 21723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3dc4b71b1004aecbba94ae63fcad0fa in term 3.
I20260806 06:41:56.646250 21494 leader_election.cc:302] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3a86a40006a4549bb96ad7e77b671a2, f3dc4b71b1004aecbba94ae63fcad0fa; no voters: 9c0c4627528642fcaf93f3f4a7f002c3
I20260806 06:41:56.646353 22261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:41:56.646456 22261 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [term 3 LEADER]: Becoming Leader. State: Replica: f3dc4b71b1004aecbba94ae63fcad0fa, State: Running, Role: LEADER
I20260806 06:41:56.646514 22261 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } }
I20260806 06:41:56.646909 22264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3dc4b71b1004aecbba94ae63fcad0fa. Latest consensus state: current_term: 3 leader_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:56.647011 22264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:56.647261 22266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:56.647763 22266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:56.647853 22266 catalog_manager.cc:1259] Loaded cluster ID: 48db753e8f4644bf9fdfbd2f9cb1b101
I20260806 06:41:56.647888 22266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:41:56.648084 22266 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:56.648172 22266 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Loaded TSK: 0
I20260806 06:41:56.648322 22266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:41:56.700443 21512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:56.702040 21512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:56.703035 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.702914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45756)
0806 06:41:56.702970 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.703004 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:41:56.703107 21723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [term 3 FOLLOWER]: Refusing update from remote peer f3dc4b71b1004aecbba94ae63fcad0fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 06:41:56.703403 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.703319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.0.0.1:45772)
0806 06:41:56.703356 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.703379 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:41:56.703405 22264 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 06:41:56.703527 21496 consensus_peers.cc:487] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa -> Peer 9c0c4627528642fcaf93f3f4a7f002c3 (127.14.51.126:43891): Couldn't send request to peer 9c0c4627528642fcaf93f3f4a7f002c3. Status: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:41:56.704078 22268 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3dc4b71b1004aecbba94ae63fcad0fa. Latest consensus state: current_term: 3 leader_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:56.704190 22268 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:56.704497 22267 mvcc.cc:205] Tried to move back new op lower bound from 7315449924413726720 to 7315449924184850432. Current Snapshot: MvccSnapshot[applied={T|T < 7315449924413726720}]
I20260806 06:41:56.704571 22264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:56.704671 22264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:56.704736 22268 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9c0c4627528642fcaf93f3f4a7f002c3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43891 } } peers { permanent_uuid: "f3dc4b71b1004aecbba94ae63fcad0fa" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41275 } attrs { promote: false } } peers { permanent_uuid: "f3a86a40006a4549bb96ad7e77b671a2" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 33857 } attrs { promote: false } } }
I20260806 06:41:56.704826 22268 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a86a40006a4549bb96ad7e77b671a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:41:56.704835 22269 mvcc.cc:205] Tried to move back new op lower bound from 7315449924413726720 to 7315449924184850432. Current Snapshot: MvccSnapshot[applied={T|T < 7315449924413726720}]
I20260806 06:41:56.709375 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42405 (local address 127.0.0.1:40702)
0806 06:41:56.706549 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.706560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:56.706613 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:56.706992 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:56.706992 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:56.706994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:56.707065 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.707067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707800 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.707801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.708530 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.708531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.709158 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.709159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.709230 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.709235 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:56.709268 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 06:41:56.709312 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":1}
I20260806 06:41:56.709699 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.14.51.66:42405)
0806 06:41:56.706880 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.706882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.706889 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:41:56.706914 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.706914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.706924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.706977 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:56.707171 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707713 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.708620 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.709067 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.709545 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.709558 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.709577 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.709617 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":78,"threads_started":1}
I20260806 06:41:56.709914 21970 tablet_service.cc:1467] Processing CreateTablet for tablet c805c4ffa01c429a9375255a40bd2b69 (DEFAULT_TABLE table=test-table [id=48dc5743468e402492ebd5ff7d95923a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:56.710060 21970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c805c4ffa01c429a9375255a40bd2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:56.710261 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37333 (local address 127.0.0.1:51696)
0806 06:41:56.706583 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.706595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:56.706649 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:56.706911 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:56.706912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:56.706914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:56.706967 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.706969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707936 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.707937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.709203 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.709205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.710002 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.710004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.710074 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.710078 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:56.710135 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:41:56.710181 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":70,"thread_start_us":88,"threads_started":1}
I20260806 06:41:56.710352 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.14.51.65:43833)
0806 06:41:56.706704 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.706706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.706767 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:41:56.706830 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.706830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.706840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.706912 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:56.707082 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707840 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.709317 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.709949 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.710026 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.710038 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.710233 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.710278 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260806 06:41:56.710306 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.14.51.67:37333)
0806 06:41:56.706767 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.706769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.706778 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:56.706805 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.706806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.706819 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.706908 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:56.707057 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707840 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.709282 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.709913 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.709984 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.709997 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.710156 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.710243 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260806 06:41:56.710563 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.706470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43833 (local address 127.0.0.1:44356)
0806 06:41:56.706702 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.706713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:56.706769 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:56.706957 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:56.706957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:56.706959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:56.706989 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.706990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.707937 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.707937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.709202 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.709205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.710094 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.710095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.710154 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.710156 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:41:56.710450 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 06:41:56.710497 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":66,"thread_start_us":68,"threads_started":1}
I20260806 06:41:56.710573 22108 tablet_service.cc:1467] Processing CreateTablet for tablet c805c4ffa01c429a9375255a40bd2b69 (DEFAULT_TABLE table=test-table [id=48dc5743468e402492ebd5ff7d95923a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:56.710691 22108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c805c4ffa01c429a9375255a40bd2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:56.710815 21831 tablet_service.cc:1467] Processing CreateTablet for tablet c805c4ffa01c429a9375255a40bd2b69 (DEFAULT_TABLE table=test-table [id=48dc5743468e402492ebd5ff7d95923a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:56.710943 21831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c805c4ffa01c429a9375255a40bd2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:56.712124 22280 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Bootstrap starting.
I20260806 06:41:56.712476 22281 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e: Bootstrap starting.
I20260806 06:41:56.712502 22280 tablet_bootstrap.cc:654] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:56.712855 22279 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Bootstrap starting.
I20260806 06:41:56.712886 22281 tablet_bootstrap.cc:654] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:56.713239 22279 tablet_bootstrap.cc:654] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:56.713541 22281 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e: No bootstrap required, opened a new log
I20260806 06:41:56.713599 22281 ts_tablet_manager.cc:1397] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:56.713824 22279 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: No bootstrap required, opened a new log
I20260806 06:41:56.713835 22280 tablet_bootstrap.cc:492] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: No bootstrap required, opened a new log
I20260806 06:41:56.713799 22281 raft_consensus.cc:348] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.713876 22281 raft_consensus.cc:374] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:56.713889 22280 ts_tablet_manager.cc:1397] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:56.713889 22279 ts_tablet_manager.cc:1397] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:56.713909 22281 raft_consensus.cc:729] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53dd49b30028417f8c5d52c954470f2e, State: Initialized, Role: FOLLOWER
I20260806 06:41:56.713981 22281 consensus_queue.cc:260] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.714025 22280 raft_consensus.cc:348] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.714047 22279 raft_consensus.cc:348] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.714103 22279 raft_consensus.cc:374] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:56.714104 22280 raft_consensus.cc:374] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:56.714118 22279 raft_consensus.cc:729] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b840b3392e4f8cac58383a1d003743, State: Initialized, Role: FOLLOWER
I20260806 06:41:56.714126 22280 raft_consensus.cc:729] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a149d9783b4f26b1282f2f38a8ac6a, State: Initialized, Role: FOLLOWER
I20260806 06:41:56.714155 22281 ts_tablet_manager.cc:1428] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:56.714171 22279 consensus_queue.cc:260] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.714169 22280 consensus_queue.cc:260] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.714349 22280 ts_tablet_manager.cc:1428] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:56.714376 22279 ts_tablet_manager.cc:1428] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:56.714550 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.714355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:41091)
0806 06:41:56.714495 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.714526 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 06:41:56.715152 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.714793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:37595)
0806 06:41:56.715065 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.715098 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260806 06:41:56.715256 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.714892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:54583)
0806 06:41:56.715196 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.715229 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
I20260806 06:41:56.715770 22038 heartbeater.cc:503] Master 127.14.51.125:41275 was elected leader, sending a full tablet report...
I20260806 06:41:56.715770 22178 heartbeater.cc:503] Master 127.14.51.125:41275 was elected leader, sending a full tablet report...
I20260806 06:41:56.716071 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.715982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:46977)
0806 06:41:56.716023 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.716045 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:41:56.716142 21899 heartbeater.cc:503] Master 127.14.51.125:41275 was elected leader, sending a full tablet report...
I20260806 06:41:56.716445 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.716365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:54409)
0806 06:41:56.716400 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.716426 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 06:41:56.716464 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.715754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:49601)
0806 06:41:56.716422 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.716445 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634}
I20260806 06:41:56.763698 22287 raft_consensus.cc:482] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:56.763782 22287 raft_consensus.cc:504] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.764122 22287 leader_election.cc:288] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333), 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
I20260806 06:41:56.768078 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.764388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:36927 (local address 127.14.51.65:43833)
0806 06:41:56.764448 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.764450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.764610 (+ 160us) server_negotiation.cc:407] Connection header received
0806 06:41:56.764698 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.764699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.764706 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.764746 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:56.764891 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.765616 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767180 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.767725 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767802 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.767812 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.767972 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.768008 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:41:56.768078 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.764465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:54185 (local address 127.14.51.67:37333)
0806 06:41:56.764514 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.764516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.764522 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:41:56.764544 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.764545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.764552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.764593 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:56.764814 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.765591 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767212 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.767733 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767802 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.767812 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.767972 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.768009 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:56.768103 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.764284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37333 (local address 127.14.51.66:54185)
0806 06:41:56.764349 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.764361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:56.764429 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:56.764646 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:56.764647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:56.764650 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:56.764717 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.764718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.765692 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.765693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.767076 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.767078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.767812 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.767890 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.767895 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:56.767954 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:41:56.768006 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":47}
I20260806 06:41:56.768103 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.764322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43833 (local address 127.14.51.66:36927)
0806 06:41:56.764530 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.764538 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:56.764625 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:56.764765 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:56.764766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:56.764768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:56.764821 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.764822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.765697 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.765697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.767074 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:56.767077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.767814 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.767814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:56.767890 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.767896 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:41:56.767954 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:41:56.768006 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":88,"thread_start_us":121,"threads_started":1}
I20260806 06:41:56.768400 22128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c805c4ffa01c429a9375255a40bd2b69" candidate_uuid: "d2b840b3392e4f8cac58383a1d003743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" is_pre_election: true
I20260806 06:41:56.768567 22128 raft_consensus.cc:2454] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2b840b3392e4f8cac58383a1d003743 in term 0.
I20260806 06:41:56.768630 21851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c805c4ffa01c429a9375255a40bd2b69" candidate_uuid: "d2b840b3392e4f8cac58383a1d003743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53dd49b30028417f8c5d52c954470f2e" is_pre_election: true
I20260806 06:41:56.768748 21851 raft_consensus.cc:2454] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2b840b3392e4f8cac58383a1d003743 in term 0.
I20260806 06:41:56.768792 21923 leader_election.cc:302] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a149d9783b4f26b1282f2f38a8ac6a, d2b840b3392e4f8cac58383a1d003743; no voters:
I20260806 06:41:56.768963 22287 raft_consensus.cc:2781] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:56.768997 22287 raft_consensus.cc:482] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:56.769011 22287 raft_consensus.cc:3037] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:56.769613 22287 raft_consensus.cc:504] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.769749 22287 leader_election.cc:288] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [CANDIDATE]: Term 1 election: Requested vote from peers 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333), 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833)
I20260806 06:41:56.769871 22128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c805c4ffa01c429a9375255a40bd2b69" candidate_uuid: "d2b840b3392e4f8cac58383a1d003743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a149d9783b4f26b1282f2f38a8ac6a"
I20260806 06:41:56.769884 21851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c805c4ffa01c429a9375255a40bd2b69" candidate_uuid: "d2b840b3392e4f8cac58383a1d003743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53dd49b30028417f8c5d52c954470f2e"
I20260806 06:41:56.769956 21851 raft_consensus.cc:3037] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:56.769956 22128 raft_consensus.cc:3037] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:56.770722 21851 raft_consensus.cc:2454] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2b840b3392e4f8cac58383a1d003743 in term 1.
I20260806 06:41:56.770896 21921 leader_election.cc:302] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53dd49b30028417f8c5d52c954470f2e, d2b840b3392e4f8cac58383a1d003743; no voters:
I20260806 06:41:56.770979 22128 raft_consensus.cc:2454] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2b840b3392e4f8cac58383a1d003743 in term 1.
I20260806 06:41:56.771008 22287 raft_consensus.cc:2781] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:56.771203 22287 raft_consensus.cc:686] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 1 LEADER]: Becoming Leader. State: Replica: d2b840b3392e4f8cac58383a1d003743, State: Running, Role: LEADER
I20260806 06:41:56.771291 22287 consensus_queue.cc:237] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } }
I20260806 06:41:56.771522 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.771428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:37535)
0806 06:41:56.771461 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.771495 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:41:56.772115 21513 catalog_manager.cc:5495] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2b840b3392e4f8cac58383a1d003743 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "d2b840b3392e4f8cac58383a1d003743" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2b840b3392e4f8cac58383a1d003743" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:56.784821 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.782232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.14.51.66:42405)
0806 06:41:56.782270 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:56.782272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:56.782298 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:41:56.782354 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:56.782355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:56.782374 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:56.782457 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:56.782606 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.783177 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.783946 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:56.784213 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:56.784285 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:56.784455 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:56.784575 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:56.784655 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:56.784703 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:41:56.784724 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:56.785931 22128 raft_consensus.cc:1261] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Refusing update from remote peer d2b840b3392e4f8cac58383a1d003743: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:56.785929 21851 raft_consensus.cc:1261] T c805c4ffa01c429a9375255a40bd2b69 P 53dd49b30028417f8c5d52c954470f2e [term 1 FOLLOWER]: Refusing update from remote peer d2b840b3392e4f8cac58383a1d003743: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:56.786226 22289 consensus_queue.cc:1035] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53dd49b30028417f8c5d52c954470f2e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:56.786353 22287 consensus_queue.cc:1035] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09a149d9783b4f26b1282f2f38a8ac6a" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:56.787040 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.786908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:39561)
0806 06:41:56.786979 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.787006 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 06:41:56.787372 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.787218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:55191)
0806 06:41:56.787307 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.787338 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 06:41:56.788460 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.788240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:53827)
0806 06:41:56.788392 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.788431 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260806 06:41:56.788621 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.788400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.65:33011)
0806 06:41:56.788532 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.788575 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 06:41:56.788692 22293 mvcc.cc:205] Tried to move back new op lower bound from 7315449924752842752 to 7315449924695695360. Current Snapshot: MvccSnapshot[applied={T|T < 7315449924752842752 or (T in {7315449924752842752})}]
I20260806 06:41:56.788975 22292 mvcc.cc:205] Tried to move back new op lower bound from 7315449924752842752 to 7315449924695695360. Current Snapshot: MvccSnapshot[applied={T|T < 7315449924752842752}]
I20260806 06:41:56.789314 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.789221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.66:38691)
0806 06:41:56.789258 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.789283 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:41:56.789587 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.789498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43891 (local address 127.14.51.67:55539)
0806 06:41:56.789533 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.789567 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 06:41:56.792145 21513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:56.792251 21513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:56.793493 21513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3dc4b71b1004aecbba94ae63fcad0fa: Sending DeleteTablet for 3 replicas of tablet c805c4ffa01c429a9375255a40bd2b69
I20260806 06:41:56.793865 21970 tablet_service.cc:1514] Processing DeleteTablet for tablet c805c4ffa01c429a9375255a40bd2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:56 UTC) from {username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:40702
I20260806 06:41:56.793882 22108 tablet_service.cc:1514] Processing DeleteTablet for tablet c805c4ffa01c429a9375255a40bd2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:56 UTC) from {username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:51696
I20260806 06:41:56.794080 22302 tablet_replica.cc:330] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: stopping tablet replica
I20260806 06:41:56.794134 22302 raft_consensus.cc:2229] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:56.794132 22303 tablet_replica.cc:330] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: stopping tablet replica
I20260806 06:41:56.794181 22303 raft_consensus.cc:2229] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:56.794215 22302 raft_consensus.cc:2258] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:56.794219 22303 raft_consensus.cc:2258] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:56.794353 21831 tablet_service.cc:1514] Processing DeleteTablet for tablet c805c4ffa01c429a9375255a40bd2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:56 UTC) from {username='oryx', principal='oryx/127.14.51.125@KRBTEST.COM'} at 127.0.0.1:44356
I20260806 06:41:56.794513 22302 ts_tablet_manager.cc:1905] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:56.794610 22303 ts_tablet_manager.cc:1905] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:56.794663 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21667
I20260806 06:41:56.796761 22302 ts_tablet_manager.cc:1918] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:56.796823 22302 log.cc:1196] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/c805c4ffa01c429a9375255a40bd2b69
I20260806 06:41:56.797050 22303 ts_tablet_manager.cc:1918] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:56.797086 22302 ts_tablet_manager.cc:1939] T c805c4ffa01c429a9375255a40bd2b69 P d2b840b3392e4f8cac58383a1d003743: Deleting consensus metadata
I20260806 06:41:56.797104 22303 log.cc:1196] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/c805c4ffa01c429a9375255a40bd2b69
I20260806 06:41:56.797392 22303 ts_tablet_manager.cc:1939] T c805c4ffa01c429a9375255a40bd2b69 P 09a149d9783b4f26b1282f2f38a8ac6a: Deleting consensus metadata
I20260806 06:41:56.797529 21499 catalog_manager.cc:4841] TS d2b840b3392e4f8cac58383a1d003743 (127.14.51.66:42405): tablet c805c4ffa01c429a9375255a40bd2b69 (table test-table [id=48dc5743468e402492ebd5ff7d95923a]) successfully deleted
I20260806 06:41:56.797816 21497 catalog_manager.cc:4841] TS 09a149d9783b4f26b1282f2f38a8ac6a (127.14.51.67:37333): tablet c805c4ffa01c429a9375255a40bd2b69 (table test-table [id=48dc5743468e402492ebd5ff7d95923a]) successfully deleted
I20260806 06:41:56.803056 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21905
I20260806 06:41:56.803081 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:56.802986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43833 (local address 127.0.0.1:44364)
0806 06:41:56.803028 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:56.803049 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:43833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:41:56.803828 21494 catalog_manager.cc:4584] TS 53dd49b30028417f8c5d52c954470f2e (127.14.51.65:43833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c805c4ffa01c429a9375255a40bd2b69: Network error: Client connection negotiation failed: client connection to 127.14.51.65:43833: connect: Connection refused (error 111)
I20260806 06:41:56.808627 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 22044
I20260806 06:41:56.815339 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21366
I20260806 06:41:56.820757 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 21570
2026-08-06T06:41:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6686 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:56 dist-test-slave-1xrh krb5kdc[22308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:56 dist-test-slave-1xrh krb5kdc[22308](info): set up 2 sockets
Aug 06 06:41:56 dist-test-slave-1xrh krb5kdc[22308](info): commencing operation
krb5kdc: starting...
W20260806 06:41:57.882750 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:41:57 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:41:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab.
Entry for principal oryx/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab.
I20260806 06:41:58.001677 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38707
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab
--principal=oryx/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:58.081724 22321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:58.081933 22321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:58.081955 22321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:58.083027 22321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:41:58.083062 22321 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:41:58.083076 22321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:58.083089 22321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:58.083101 22321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:41:58.083113 22321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:41:58.084689 22321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.126.keytab
--principal=oryx/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:58.084874 22321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:58.085038 22321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:58.089649 22326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:58.089697 22327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.089885 22321 server_base.cc:1034] running on GCE node
W20260806 06:41:58.089763 22329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.090183 22321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:58.090462 22321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:58.091620 22321 hybrid_clock.cc:648] HybridClock initialized: now 1785998518091607 us; error 42 us; skew 500 ppm
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:58.093992 22321 init.cc:377] Logged in from keytab as oryx/127.14.51.126@KRBTEST.COM (short username oryx)
I20260806 06:41:58.094420 22321 webserver.cc:460] Webserver started at http://127.14.51.126:38639/ using document root <none> and password file <none>
I20260806 06:41:58.094627 22321 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:58.094666 22321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:58.094769 22321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:58.095790 22321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "42a5a3805f9844439862437760cf8315"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "2d4c4b2c499cb804e0cc146b35b4c3d9"
server_key_iv: "945bb0f34f395cacf5320acef799d62c"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.096099 22321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "42a5a3805f9844439862437760cf8315"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "2d4c4b2c499cb804e0cc146b35b4c3d9"
server_key_iv: "945bb0f34f395cacf5320acef799d62c"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.097349 22321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:58.097965 22336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.098134 22321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:58.098218 22321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "42a5a3805f9844439862437760cf8315"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "2d4c4b2c499cb804e0cc146b35b4c3d9"
server_key_iv: "945bb0f34f395cacf5320acef799d62c"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.098273 22321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:58.112412 22321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:58.114131 22321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:58.114248 22321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:58.118852 22321 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38707
I20260806 06:41:58.118927 22400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38707 every 8 connection(s)
I20260806 06:41:58.119271 22321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:41:58.119969 22401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:58.122509 22401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Bootstrap starting.
I20260806 06:41:58.123205 22401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:58.123556 22401 log.cc:824] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:58.124220 22401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: No bootstrap required, opened a new log
I20260806 06:41:58.125888 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 22321
I20260806 06:41:58.125965 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:41:58.126153 14541 external_mini_cluster.cc:1428] Setting key 0766610663b6922ecae63e411f9ee9f3
I20260806 06:41:58.126398 22401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } }
I20260806 06:41:58.126585 22401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:58.126652 22401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a5a3805f9844439862437760cf8315, State: Initialized, Role: FOLLOWER
I20260806 06:41:58.126788 22401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } }
I20260806 06:41:58.126883 22401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:41:58.126938 22401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:41:58.126991 22401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:58.127553 22401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } }
I20260806 06:41:58.127668 22401 leader_election.cc:302] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a5a3805f9844439862437760cf8315; no voters:
I20260806 06:41:58.127827 22401 leader_election.cc:288] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:41:58.127861 22406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:58.128085 22406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [term 1 LEADER]: Becoming Leader. State: Replica: 42a5a3805f9844439862437760cf8315, State: Running, Role: LEADER
I20260806 06:41:58.128177 22401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:41:58.128213 22406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } }
I20260806 06:41:58.128641 22406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a5a3805f9844439862437760cf8315. Latest consensus state: current_term: 1 leader_uuid: "42a5a3805f9844439862437760cf8315" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } } }
I20260806 06:41:58.128732 22406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:58.128880 22407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42a5a3805f9844439862437760cf8315" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a5a3805f9844439862437760cf8315" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38707 } } }
I20260806 06:41:58.128950 22407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315 [sys.catalog]: This master's current role is: LEADER
I20260806 06:41:58.129056 22415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:41:58.129594 22415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:41:58.131044 22415 catalog_manager.cc:1347] Generated new cluster ID: 370131b009ac45819c6954477900d7ce
I20260806 06:41:58.131101 22415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:58.137034 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.126652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.14.51.126:38707)
0806 06:41:58.126841 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.126850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.126879 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:41:58.127008 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.127015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.127069 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.127232 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.127912 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.129236 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.130402 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.130896 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.131015 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.133288 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.133292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.133292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.133303 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.135550 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.135902 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.135905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.135906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.135998 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.136201 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.136202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.136202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.136427 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.136594 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.136662 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.136891 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260806 06:41:58.141460 22415 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:41:58.142185 22415 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:41:58.148620 22415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Generated new TSK 0
I20260806 06:41:58.148772 22415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
I20260806 06:41:58.177699 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38707
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:58.257686 22427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:58.257929 22427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:58.257954 22427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:58.259097 22427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:58.259161 22427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:58.259181 22427 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:58.259198 22427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:41:58.260629 22427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:58.260838 22427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:58.261039 22427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:58.265842 22433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:58.265853 22435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.265902 22427 server_base.cc:1034] running on GCE node
W20260806 06:41:58.265849 22432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.266211 22427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:58.266445 22427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:58.267584 22427 hybrid_clock.cc:648] HybridClock initialized: now 1785998518267582 us; error 33 us; skew 500 ppm
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:58.270125 22427 init.cc:377] Logged in from keytab as oryx/127.14.51.65@KRBTEST.COM (short username oryx)
I20260806 06:41:58.270581 22427 webserver.cc:460] Webserver started at http://127.14.51.65:33007/ using document root <none> and password file <none>
I20260806 06:41:58.270807 22427 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:58.270888 22427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:58.271031 22427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:58.272085 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "4b5ac674774268e5c181d7377611435e"
server_key_iv: "7f12269f620fc640057b1666e9d55a2b"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.272456 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "4b5ac674774268e5c181d7377611435e"
server_key_iv: "7f12269f620fc640057b1666e9d55a2b"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.273727 22427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:58.274350 22442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.274622 22427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:58.274716 22427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "4b5ac674774268e5c181d7377611435e"
server_key_iv: "7f12269f620fc640057b1666e9d55a2b"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.274783 22427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:58.314752 22427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:58.317147 22427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:58.317333 22427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:58.317871 22427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:58.317936 22427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.317975 22427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:58.318013 22427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:58.325078 22427 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:38627
I20260806 06:41:58.325162 22559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:38627 every 8 connection(s)
I20260806 06:41:58.325511 22427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:41:58.327805 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.318370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:57351 (local address 127.14.51.126:38707)
0806 06:41:58.318420 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.318426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.318783 (+ 357us) server_negotiation.cc:407] Connection header received
0806 06:41:58.319438 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.319439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.319458 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.319511 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.320362 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.321058 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.322077 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.322286 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.322369 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.324907 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.324909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.324909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.324920 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.326441 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.326841 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.326843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.326843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.326897 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.327160 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.327161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.327161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.327273 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.327368 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.327633 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.327714 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:41:58.328006 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.318446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.65:57351)
0806 06:41:58.318627 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.318680 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.319324 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.319592 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.319596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.319977 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.320266 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.320272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.321149 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.321150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.321968 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.321970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.322392 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.322393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.322519 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.323064 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.323073 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.324803 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.326532 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.326539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.326541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.326751 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.326964 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.326965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.326965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.327071 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.327370 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.327377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.327508 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.327612 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.327762 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 06:41:58.328471 22560 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38707
I20260806 06:41:58.328594 22560 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:58.328825 22560 heartbeater.cc:511] Master 127.14.51.126:38707 requested a full tablet report, sending...
I20260806 06:41:58.329429 22352 ts_manager.cc:194] Registered new tserver with Master: 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627)
I20260806 06:41:58.330372 22352 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:57351
I20260806 06:41:58.332962 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 22427
I20260806 06:41:58.333087 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:41:58.333362 14541 external_mini_cluster.cc:1428] Setting key 6170ec5e5d6842cfebabfd1d5c3b6974
I20260806 06:41:58.335414 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.329490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.65:43453)
0806 06:41:58.329745 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.329756 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.329879 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.330014 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.330015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.330197 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.330264 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.330266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.331212 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.331213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.331681 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.331682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.331994 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.331995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.332039 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.332454 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.332462 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.332711 (+ 249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.334453 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.334453 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.334454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.334623 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.334841 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.334842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.334842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.334892 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.335165 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.335166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.335211 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.335262 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.335336 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":187,"threads_started":1}
I20260806 06:41:58.335538 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.329543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43453 (local address 127.14.51.126:38707)
0806 06:41:58.329586 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.329587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.329803 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:41:58.329937 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.329938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.329951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.329997 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.330322 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.331050 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.331764 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.331939 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.331985 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.332792 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.332794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.332794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.332801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.334382 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.334689 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.334690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.334690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.334748 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.334945 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.334946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.334947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.335073 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.335160 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.335302 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.335405 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
I20260806 06:41:58.358510 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38707
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:58.439451 22570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:58.439693 22570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:58.439728 22570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:58.440891 22570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:58.440954 22570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:58.440989 22570 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:58.441035 22570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:41:58.442588 22570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:58.442832 22570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:58.443068 22570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:58.447976 22577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:58.447993 22575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.448032 22570 server_base.cc:1034] running on GCE node
W20260806 06:41:58.447984 22579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.448378 22570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:58.448621 22570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:58.449769 22570 hybrid_clock.cc:648] HybridClock initialized: now 1785998518449763 us; error 36 us; skew 500 ppm
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:58.452342 22570 init.cc:377] Logged in from keytab as oryx/127.14.51.66@KRBTEST.COM (short username oryx)
I20260806 06:41:58.452777 22570 webserver.cc:460] Webserver started at http://127.14.51.66:41597/ using document root <none> and password file <none>
I20260806 06:41:58.453003 22570 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:58.453079 22570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:58.453225 22570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:58.454180 22570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "165da6ab00e84028ae02aec52964da60"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "c2e497dcb8ad50ac6f60bf591fb411c4"
server_key_iv: "9eb97782610ac1c2ffe8f0d668f34f1e"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.454533 22570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "165da6ab00e84028ae02aec52964da60"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "c2e497dcb8ad50ac6f60bf591fb411c4"
server_key_iv: "9eb97782610ac1c2ffe8f0d668f34f1e"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.455895 22570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:41:58.456579 22586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.456753 22570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:58.456845 22570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "165da6ab00e84028ae02aec52964da60"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "c2e497dcb8ad50ac6f60bf591fb411c4"
server_key_iv: "9eb97782610ac1c2ffe8f0d668f34f1e"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.456919 22570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:58.492393 22570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:58.494264 22570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:58.494459 22570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:58.495028 22570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:58.495082 22570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.495123 22570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:58.495189 22570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:58.502643 22570 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:43387
I20260806 06:41:58.502918 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:43387 every 8 connection(s)
I20260806 06:41:58.503039 22570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:41:58.504992 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.495461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:43995 (local address 127.14.51.126:38707)
0806 06:41:58.495528 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.495531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.495819 (+ 288us) server_negotiation.cc:407] Connection header received
0806 06:41:58.496584 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.496586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.496599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.496654 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.497626 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.498124 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.499149 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.499355 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.499440 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.502347 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.502349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.502349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.502356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.503937 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.504289 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.504290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.504290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.504338 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.504513 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.504513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.504513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.504612 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.504662 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.504874 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.504929 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:41:58.505154 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.495464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.66:43995)
0806 06:41:58.495659 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.495716 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.496455 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.496755 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.496762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.497194 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.497517 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.497524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.498221 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.498222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.499020 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.499022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.499451 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.499452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.499569 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.500048 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.500056 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.502259 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.504010 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.504015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.504016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.504225 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.504379 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.504379 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.504380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.504466 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.504666 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.504669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.504773 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.504859 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.504972 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260806 06:41:58.505611 22703 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38707
I20260806 06:41:58.505700 22703 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:58.505878 22703 heartbeater.cc:511] Master 127.14.51.126:38707 requested a full tablet report, sending...
I20260806 06:41:58.506294 22353 ts_manager.cc:194] Registered new tserver with Master: 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387)
I20260806 06:41:58.506776 22353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:43995
I20260806 06:41:58.511586 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.506626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:51083 (local address 127.14.51.126:38707)
0806 06:41:58.506920 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.506921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.506929 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:58.506947 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.506947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.506956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.507004 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.507388 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.507832 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.508489 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.508689 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.508756 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.509503 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.509504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.509505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.509511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.510484 (+ 973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.510778 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.510779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.510780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.510828 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.511014 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.511015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.511015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.511222 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.511305 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.511439 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.511508 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260806 06:41:58.511615 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.506542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.66:51083)
0806 06:41:58.506663 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.506673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.506723 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.507009 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.507010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.507256 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.507313 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.507315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.507920 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.507921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.508410 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.508412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.508756 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.508757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.508837 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.509176 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.509180 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.509420 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.510560 (+ 1140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.510561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.510561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.510715 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.510898 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.510899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.510899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.510948 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.511298 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.511298 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.511339 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.511416 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.511508 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260806 06:41:58.513145 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 22570
I20260806 06:41:58.513238 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:41:58.513388 14541 external_mini_cluster.cc:1428] Setting key e8cebdf692877a86454a9573359e3bee
WARNING: no policy specified for oryx/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
I20260806 06:41:58.538749 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38707
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:41:58.618762 22712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:41:58.619040 22712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:41:58.619066 22712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:41:58.620190 22712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:41:58.620231 22712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:41:58.620251 22712 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:41:58.620267 22712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:41:58.621717 22712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.51.126:38707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:41:58.621917 22712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:41:58.622088 22712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:41:58.626984 22718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.627071 22712 server_base.cc:1034] running on GCE node
W20260806 06:41:58.626937 22717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:41:58.627283 22720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:41:58.627538 22712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:41:58.627730 22712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:41:58.628891 22712 hybrid_clock.cc:648] HybridClock initialized: now 1785998518628887 us; error 31 us; skew 500 ppm
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:41:58.631552 22712 init.cc:377] Logged in from keytab as oryx/127.14.51.67@KRBTEST.COM (short username oryx)
I20260806 06:41:58.631999 22712 webserver.cc:460] Webserver started at http://127.14.51.67:38873/ using document root <none> and password file <none>
I20260806 06:41:58.632189 22712 fs_manager.cc:359] Metadata directory not provided
I20260806 06:41:58.632225 22712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:41:58.632306 22712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:41:58.633244 22712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "2e1907e616574d9cb83d9b98bff87327"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "bcd457aa0f70fce76520821be757bb66"
server_key_iv: "bcce31bd824e311c2c24d1aaf601fc4f"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.633538 22712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "2e1907e616574d9cb83d9b98bff87327"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "bcd457aa0f70fce76520821be757bb66"
server_key_iv: "bcce31bd824e311c2c24d1aaf601fc4f"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.634750 22712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:41:58.635510 22727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.635656 22712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.635733 22712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "2e1907e616574d9cb83d9b98bff87327"
format_stamp: "Formatted at 2026-08-06 06:41:58 on dist-test-slave-1xrh"
server_key: "bcd457aa0f70fce76520821be757bb66"
server_key_iv: "bcce31bd824e311c2c24d1aaf601fc4f"
server_key_version: "encryptionkey@0"
I20260806 06:41:58.635784 22712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:41:58.648617 22712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:41:58.650373 22712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:41:58.650542 22712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:41:58.651003 22712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:41:58.651042 22712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:58.651062 22712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:41:58.651074 22712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:58.658097 22712 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:36353
I20260806 06:41:58.658178 22849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:36353 every 8 connection(s)
I20260806 06:41:58.658458 22712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:41:58.661376 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.651467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:40387 (local address 127.14.51.126:38707)
0806 06:41:58.651537 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.651539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.651748 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:41:58.652424 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.652429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.652439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.652481 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.653451 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.654247 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.655665 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.656009 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.656077 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.658555 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.658557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.658557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.658565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.659981 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.660401 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.660402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.660403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.660460 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.660763 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.660764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.660764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.660887 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.660959 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.661209 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.661287 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:41:58.661552 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.651442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.67:40387)
0806 06:41:58.651611 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.651657 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.652303 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.652556 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.652560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.653044 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.653347 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.653354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.654328 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.654329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.655509 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.655511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.656094 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.656095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.656219 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.656736 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.656747 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.658451 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.660087 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.660093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.660095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.660313 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.660522 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.660574 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.660575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.660697 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.660950 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.660956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.661085 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.661186 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.661329 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260806 06:41:58.662056 22850 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38707
I20260806 06:41:58.662134 22850 heartbeater.cc:464] Registering TS with master...
I20260806 06:41:58.662300 22850 heartbeater.cc:511] Master 127.14.51.126:38707 requested a full tablet report, sending...
I20260806 06:41:58.662683 22352 ts_manager.cc:194] Registered new tserver with Master: 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:58.663065 22352 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:40387
I20260806 06:41:58.663297 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 22712
I20260806 06:41:58.663403 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:41:58.663570 14541 external_mini_cluster.cc:1428] Setting key 96fe7d80255ad6cd4f0aa831cd7d914c
I20260806 06:41:58.664440 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:41:58.669378 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.663057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:58425 (local address 127.14.51.126:38707)
0806 06:41:58.663110 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.663112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.663829 (+ 717us) server_negotiation.cc:407] Connection header received
0806 06:41:58.663925 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.663926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.663937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.663987 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.664360 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.664874 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.665860 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.666049 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.666094 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.666966 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.666968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.666968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.666975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.668338 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.668743 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.668744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.668744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.668790 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.668988 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.668989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.668989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.669069 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.669113 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.669272 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.669323 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:41:58.669520 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.663003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.14.51.67:58425)
0806 06:41:58.663744 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.663754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.663839 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.664000 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.664001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.664215 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:58.664272 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.664274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.664975 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.664976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.665790 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.665793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.666114 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.666115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.666183 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.666606 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:58.666610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:58.666873 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:58.668421 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.668422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.668422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.668667 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.668852 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:58.668854 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:58.668854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:58.668925 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:58.669127 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:58.669128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:58.669175 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.669260 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.669362 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":74,"threads_started":1}
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:41:58.685685 22405 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:58.680786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36610 (local address 127.14.51.126:38707)
0806 06:41:58.680825 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.680826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.680861 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:41:58.680935 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.680935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.680943 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.680989 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.681508 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.681941 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.682579 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.682739 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.682783 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.685604 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36610: BlockingRecv error: recv got EOF from 127.0.0.1:36610 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 06:41:58 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:58.692407 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.686344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.14.51.126:38707)
0806 06:41:58.686401 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.686403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.686494 (+ 91us) server_negotiation.cc:407] Connection header received
0806 06:41:58.686565 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.686566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.686575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.686641 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:58.687071 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.687575 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.688229 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.688414 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.688494 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.690401 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:58.690402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:58.690402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:58.690408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:58.691607 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.691894 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.691894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.691895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.691940 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:58.692077 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:58.692077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:58.692077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:58.692155 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:58.692218 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.692292 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.692343 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:41:58.693904 22352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:58.695528 22352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:58.701992 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:43387 (local address 127.0.0.1:34940)
0806 06:41:58.698140 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.698147 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.698201 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.698433 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.698434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.698437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:58.698482 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.698484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699360 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.699362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.700716 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.700718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701691 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.701767 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.701782 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.701861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.701913 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":67,"threads_started":1}
I20260806 06:41:58.702113 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:36353 (local address 127.0.0.1:58344)
0806 06:41:58.697927 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.697966 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.698034 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.698392 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.698393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.698394 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:58.698433 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.698435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699726 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.699727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.700832 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.700833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701794 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.701928 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.701933 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.701985 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.702028 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260806 06:41:58.702244 22873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:38627 (local address 127.0.0.1:58470)
0806 06:41:58.697990 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.698006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.698090 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.698346 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.698349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.698351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:58.698411 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.698412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699742 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.699743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.701011 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.701012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701979 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.702054 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.702058 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.702115 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.702156 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":414,"thread_start_us":94,"threads_started":1}
I20260806 06:41:58.702278 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.14.51.66:43387)
0806 06:41:58.698038 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.698042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.698199 (+ 157us) server_negotiation.cc:407] Connection header received
0806 06:41:58.698265 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.698267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.698297 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.698368 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:58.698561 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699283 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.700805 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701582 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701687 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.701762 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.701884 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.701927 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260806 06:41:58.702371 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.14.51.67:36353)
0806 06:41:58.698209 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.698213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.698221 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:58.698253 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.698255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.698268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.698336 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:58.698510 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699626 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.700901 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701620 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701682 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.701756 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.702219 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.702262 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":98,"threads_started":1}
I20260806 06:41:58.702555 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.697746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58470 (local address 127.14.51.65:38627)
0806 06:41:58.697930 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.697934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.698079 (+ 145us) server_negotiation.cc:407] Connection header received
0806 06:41:58.698160 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.698163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.698179 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.698258 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:58.698498 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.699658 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701099 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.701800 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.701865 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.701944 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.702408 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.702461 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260806 06:41:58.702926 22636 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec7dc62eda476ebe27990241ce4172 (DEFAULT_TABLE table=test-table [id=6c83848351fa40a488796fd80b9cd6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:58.703012 22781 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec7dc62eda476ebe27990241ce4172 (DEFAULT_TABLE table=test-table [id=6c83848351fa40a488796fd80b9cd6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:58.703060 22493 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec7dc62eda476ebe27990241ce4172 (DEFAULT_TABLE table=test-table [id=6c83848351fa40a488796fd80b9cd6be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:58.703325 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec7dc62eda476ebe27990241ce4172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:58.703323 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec7dc62eda476ebe27990241ce4172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:58.703471 22493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec7dc62eda476ebe27990241ce4172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:58.706447 22878 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Bootstrap starting.
I20260806 06:41:58.706951 22879 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Bootstrap starting.
I20260806 06:41:58.707284 22878 tablet_bootstrap.cc:654] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:58.707652 22879 tablet_bootstrap.cc:654] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:58.707669 22878 log.cc:824] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:58.707978 22879 log.cc:824] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:58.708267 22880 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Bootstrap starting.
I20260806 06:41:58.708470 22878 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: No bootstrap required, opened a new log
I20260806 06:41:58.708549 22878 ts_tablet_manager.cc:1397] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 06:41:58.708860 22880 tablet_bootstrap.cc:654] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:58.709178 22880 log.cc:824] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Log is configured to *not* fsync() on all Append() calls
I20260806 06:41:58.709889 22880 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: No bootstrap required, opened a new log
I20260806 06:41:58.709964 22880 ts_tablet_manager.cc:1397] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:58.710642 22878 raft_consensus.cc:348] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.710809 22878 raft_consensus.cc:374] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:58.710846 22878 raft_consensus.cc:729] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef222ac6c1e41b38a180526dbcaa01a, State: Initialized, Role: FOLLOWER
I20260806 06:41:58.710975 22878 consensus_queue.cc:260] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.711263 22878 ts_tablet_manager.cc:1428] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:41:58.711390 22560 heartbeater.cc:503] Master 127.14.51.126:38707 was elected leader, sending a full tablet report...
I20260806 06:41:58.711711 22879 tablet_bootstrap.cc:492] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: No bootstrap required, opened a new log
I20260806 06:41:58.711782 22879 ts_tablet_manager.cc:1397] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:41:58.712057 22880 raft_consensus.cc:348] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.712226 22880 raft_consensus.cc:374] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:58.712273 22880 raft_consensus.cc:729] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Initialized, Role: FOLLOWER
I20260806 06:41:58.712384 22880 consensus_queue.cc:260] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.712638 22880 ts_tablet_manager.cc:1428] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:41:58.712721 22703 heartbeater.cc:503] Master 127.14.51.126:38707 was elected leader, sending a full tablet report...
I20260806 06:41:58.713744 22879 raft_consensus.cc:348] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.713858 22879 raft_consensus.cc:374] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:58.713878 22879 raft_consensus.cc:729] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Initialized, Role: FOLLOWER
I20260806 06:41:58.713968 22879 consensus_queue.cc:260] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.714183 22879 ts_tablet_manager.cc:1428] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:41:58.714242 22850 heartbeater.cc:503] Master 127.14.51.126:38707 was elected leader, sending a full tablet report...
W20260806 06:41:58.753719 22704 tablet.cc:2369] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:58.826458 22561 tablet.cc:2369] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:41:58.909359 22851 tablet.cc:2369] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:41:58.918682 22886 raft_consensus.cc:482] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:58.918810 22886 raft_consensus.cc:504] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.919229 22886 leader_election.cc:288] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387)
I20260806 06:41:58.922217 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.919545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:37813 (local address 127.14.51.65:38627)
0806 06:41:58.919586 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.919588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.919676 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:41:58.919794 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.919795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.919804 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.919952 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:58.920119 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.920600 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921495 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.921846 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921927 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.921936 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.922097 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.922134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:58.922267 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.919416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:38627 (local address 127.14.51.67:37813)
0806 06:41:58.919518 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.919603 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.919703 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.919961 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.919962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.919964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:58.920042 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.920044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.920679 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.920680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.921407 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.921408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.921930 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.922014 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.922017 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.922088 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.922127 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 06:41:58.922277 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.919618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:45809 (local address 127.14.51.66:43387)
0806 06:41:58.919683 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.919686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.919896 (+ 210us) server_negotiation.cc:407] Connection header received
0806 06:41:58.919972 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.919973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.919982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.920028 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:58.920185 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.920640 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921538 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.921893 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921983 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.921990 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.922144 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.922192 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:41:58.922391 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.919622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:43387 (local address 127.14.51.67:45809)
0806 06:41:58.919805 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:58.919814 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:58.919905 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:58.920028 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:58.920029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:58.920030 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:58.920117 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.920119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.920721 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.920722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.921451 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:58.921452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.921984 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.921985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:58.922063 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.922066 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:58.922125 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:41:58.922165 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":130,"threads_started":1}
I20260806 06:41:58.922734 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec7dc62eda476ebe27990241ce4172" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" is_pre_election: true
I20260806 06:41:58.922792 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec7dc62eda476ebe27990241ce4172" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165da6ab00e84028ae02aec52964da60" is_pre_election: true
I20260806 06:41:58.922902 22513 raft_consensus.cc:2454] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 0.
I20260806 06:41:58.922982 22656 raft_consensus.cc:2454] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 0.
I20260806 06:41:58.923154 22732 leader_election.cc:302] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef222ac6c1e41b38a180526dbcaa01a, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:58.923313 22886 raft_consensus.cc:2781] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:58.923396 22886 raft_consensus.cc:482] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:58.923432 22886 raft_consensus.cc:3037] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:58.924475 22886 raft_consensus.cc:504] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.924626 22886 leader_election.cc:288] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387)
I20260806 06:41:58.924778 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec7dc62eda476ebe27990241ce4172" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165da6ab00e84028ae02aec52964da60"
I20260806 06:41:58.924780 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec7dc62eda476ebe27990241ce4172" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
I20260806 06:41:58.924846 22656 raft_consensus.cc:3037] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:58.924866 22513 raft_consensus.cc:3037] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:58.925539 22656 raft_consensus.cc:2454] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 1.
I20260806 06:41:58.925550 22513 raft_consensus.cc:2454] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 1.
I20260806 06:41:58.925698 22731 leader_election.cc:302] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165da6ab00e84028ae02aec52964da60, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:58.925846 22886 raft_consensus.cc:2781] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:58.925992 22886 raft_consensus.cc:686] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 1 LEADER]: Becoming Leader. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Running, Role: LEADER
I20260806 06:41:58.926079 22886 consensus_queue.cc:237] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } }
I20260806 06:41:58.926795 22352 catalog_manager.cc:5495] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67). New cstate: current_term: 1 leader_uuid: "2e1907e616574d9cb83d9b98bff87327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:58.987039 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:58.984100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.14.51.67:36353)
0806 06:41:58.984165 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:58.984167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:58.984177 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:41:58.984223 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:58.984224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:58.984246 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:58.984330 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:58.984495 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.985232 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.986256 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:58.986459 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:58.986538 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:58.986714 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:58.986826 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:58.986920 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:58.986936 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 06:41:58.986956 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:41:58.989034 22513 raft_consensus.cc:1261] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Refusing update from remote peer 2e1907e616574d9cb83d9b98bff87327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:58.989034 22656 raft_consensus.cc:1261] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Refusing update from remote peer 2e1907e616574d9cb83d9b98bff87327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:58.989456 22888 consensus_queue.cc:1035] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:58.989600 22886 consensus_queue.cc:1035] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:58.992167 22892 mvcc.cc:205] Tried to move back new op lower bound from 7315449933776097280 to 7315449933521842176. Current Snapshot: MvccSnapshot[applied={T|T < 7315449933776097280}]
I20260806 06:41:58.993047 22893 mvcc.cc:205] Tried to move back new op lower bound from 7315449933776097280 to 7315449933521842176. Current Snapshot: MvccSnapshot[applied={T|T < 7315449933776097280 or (T in {7315449933776097280})}]
I20260806 06:41:58.994607 22353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:58.994697 22353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:58.995455 22353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Sending DeleteTablet for 3 replicas of tablet 28ec7dc62eda476ebe27990241ce4172
I20260806 06:41:58.995810 22493 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ec7dc62eda476ebe27990241ce4172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:58 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58470
I20260806 06:41:58.995867 22636 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ec7dc62eda476ebe27990241ce4172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:58 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34940
I20260806 06:41:58.995968 22781 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ec7dc62eda476ebe27990241ce4172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:58 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58344
I20260806 06:41:58.996153 22902 tablet_replica.cc:330] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: stopping tablet replica
I20260806 06:41:58.996286 22902 raft_consensus.cc:2229] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:58.996340 22903 tablet_replica.cc:330] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: stopping tablet replica
I20260806 06:41:58.996443 22903 raft_consensus.cc:2229] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:58.996382 22353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:58.996523 22353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:58.996552 22903 raft_consensus.cc:2258] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:58.996799 22902 raft_consensus.cc:2258] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:58.997011 22903 ts_tablet_manager.cc:1905] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:58.997439 22902 ts_tablet_manager.cc:1905] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:58.997869 22636 tablet_service.cc:1467] Processing CreateTablet for tablet 3eae7ad55340454f8578529735e6a472 (DEFAULT_TABLE table=test-table [id=d8906ed1f5854bcab49eed3a69e5d020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:58.998029 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eae7ad55340454f8578529735e6a472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:58.998867 22903 ts_tablet_manager.cc:1918] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:58.998948 22903 log.cc:1196] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/28ec7dc62eda476ebe27990241ce4172
I20260806 06:41:58.999233 22903 ts_tablet_manager.cc:1939] T 28ec7dc62eda476ebe27990241ce4172 P 165da6ab00e84028ae02aec52964da60: Deleting consensus metadata
I20260806 06:41:58.999476 22880 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Bootstrap starting.
I20260806 06:41:58.999480 22902 ts_tablet_manager.cc:1918] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:58.999620 22902 log.cc:1196] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/28ec7dc62eda476ebe27990241ce4172
I20260806 06:41:58.999894 22880 tablet_bootstrap.cc:654] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.000002 22902 ts_tablet_manager.cc:1939] T 28ec7dc62eda476ebe27990241ce4172 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting consensus metadata
I20260806 06:41:59.000468 22339 catalog_manager.cc:4841] TS 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387): tablet 28ec7dc62eda476ebe27990241ce4172 (table test-table [id=6c83848351fa40a488796fd80b9cd6be]) successfully deleted
I20260806 06:41:59.000458 22493 tablet_service.cc:1467] Processing CreateTablet for tablet 3eae7ad55340454f8578529735e6a472 (DEFAULT_TABLE table=test-table [id=d8906ed1f5854bcab49eed3a69e5d020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.000573 22493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eae7ad55340454f8578529735e6a472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.000556 22340 catalog_manager.cc:4841] TS 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627): tablet 28ec7dc62eda476ebe27990241ce4172 (table test-table [id=6c83848351fa40a488796fd80b9cd6be]) successfully deleted
I20260806 06:41:59.000840 22781 tablet_service.cc:1467] Processing CreateTablet for tablet 3eae7ad55340454f8578529735e6a472 (DEFAULT_TABLE table=test-table [id=d8906ed1f5854bcab49eed3a69e5d020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.000975 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eae7ad55340454f8578529735e6a472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.001797 22878 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Bootstrap starting.
I20260806 06:41:59.002167 22878 tablet_bootstrap.cc:654] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.002475 22879 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Bootstrap starting.
I20260806 06:41:59.002887 22880 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: No bootstrap required, opened a new log
I20260806 06:41:59.002935 22879 tablet_bootstrap.cc:654] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.002954 22880 ts_tablet_manager.cc:1397] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:41:59.003100 22880 raft_consensus.cc:348] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.003213 22880 raft_consensus.cc:374] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.003243 22880 raft_consensus.cc:729] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.003288 22880 consensus_queue.cc:260] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.003391 22880 ts_tablet_manager.cc:1428] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:59.003613 22879 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: No bootstrap required, opened a new log
I20260806 06:41:59.003664 22879 ts_tablet_manager.cc:1397] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:59.003705 22878 tablet_bootstrap.cc:492] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: No bootstrap required, opened a new log
I20260806 06:41:59.003746 22878 ts_tablet_manager.cc:1397] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:41:59.003793 22879 raft_consensus.cc:348] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.003878 22879 raft_consensus.cc:374] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.003899 22879 raft_consensus.cc:729] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.003896 22878 raft_consensus.cc:348] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.003942 22879 consensus_queue.cc:260] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.003963 22878 raft_consensus.cc:374] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.003988 22878 raft_consensus.cc:729] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef222ac6c1e41b38a180526dbcaa01a, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.004024 22879 ts_tablet_manager.cc:1428] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:59.004033 22878 consensus_queue.cc:260] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.004346 22878 ts_tablet_manager.cc:1428] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:59.005235 22905 tablet_replica.cc:330] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: stopping tablet replica
I20260806 06:41:59.005339 22905 raft_consensus.cc:2229] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:59.005750 22905 raft_consensus.cc:2258] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.006388 22905 ts_tablet_manager.cc:1905] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.008562 22905 ts_tablet_manager.cc:1918] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:59.008643 22905 log.cc:1196] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/28ec7dc62eda476ebe27990241ce4172
I20260806 06:41:59.008946 22905 ts_tablet_manager.cc:1939] T 28ec7dc62eda476ebe27990241ce4172 P 2e1907e616574d9cb83d9b98bff87327: Deleting consensus metadata
I20260806 06:41:59.009357 22337 catalog_manager.cc:4841] TS 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353): tablet 28ec7dc62eda476ebe27990241ce4172 (table test-table [id=6c83848351fa40a488796fd80b9cd6be]) successfully deleted
I20260806 06:41:59.118949 22885 raft_consensus.cc:482] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:59.119081 22885 raft_consensus.cc:504] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.119475 22885 leader_election.cc:288] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.122813 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.119868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:42137 (local address 127.14.51.67:36353)
0806 06:41:59.119912 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.119914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.120040 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:41:59.120098 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.120099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.120109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.120163 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:59.120300 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.121065 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122114 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.122478 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122557 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.122566 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.122707 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.122743 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:41:59.122869 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.119645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:36353 (local address 127.14.51.66:42137)
0806 06:41:59.119958 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.119968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.120039 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.120168 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.120169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.120172 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:59.120246 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.120247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.121144 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.121145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.122029 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.122031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.122565 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.122637 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.122640 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.122700 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.122741 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":36,"thread_start_us":61,"threads_started":1}
I20260806 06:41:59.122852 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.119891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:48115 (local address 127.14.51.65:38627)
0806 06:41:59.119944 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.119946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.119954 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:59.119983 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.119984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.119992 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.120054 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:41:59.120255 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.121020 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122138 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.122498 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122583 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.122592 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.122736 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.122778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:41:59.123081 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.119633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:38627 (local address 127.14.51.66:48115)
0806 06:41:59.119766 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.119779 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.119832 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.120109 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.120111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.120113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:41:59.120182 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.120184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.121119 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.121121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.122038 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.122040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.122584 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.122585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.122660 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.122664 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.122727 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.122772 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":53,"thread_start_us":62,"threads_started":1}
I20260806 06:41:59.123087 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eae7ad55340454f8578529735e6a472" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327" is_pre_election: true
I20260806 06:41:59.123270 22801 raft_consensus.cc:2454] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.123385 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eae7ad55340454f8578529735e6a472" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" is_pre_election: true
I20260806 06:41:59.123445 22587 leader_election.cc:302] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165da6ab00e84028ae02aec52964da60, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:59.123495 22513 raft_consensus.cc:2454] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.123566 22885 raft_consensus.cc:2781] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:59.123603 22885 raft_consensus.cc:482] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:59.123625 22885 raft_consensus.cc:3037] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.124269 22885 raft_consensus.cc:504] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.124369 22885 leader_election.cc:288] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.124531 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eae7ad55340454f8578529735e6a472" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
I20260806 06:41:59.124534 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eae7ad55340454f8578529735e6a472" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327"
I20260806 06:41:59.124603 22513 raft_consensus.cc:3037] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.124617 22801 raft_consensus.cc:3037] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.125226 22801 raft_consensus.cc:2454] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.125262 22513 raft_consensus.cc:2454] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.125424 22590 leader_election.cc:302] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef222ac6c1e41b38a180526dbcaa01a, 165da6ab00e84028ae02aec52964da60; no voters:
I20260806 06:41:59.125510 22885 raft_consensus.cc:2781] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:59.125630 22885 raft_consensus.cc:686] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 1 LEADER]: Becoming Leader. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Running, Role: LEADER
I20260806 06:41:59.125732 22885 consensus_queue.cc:237] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.126377 22353 catalog_manager.cc:5495] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 165da6ab00e84028ae02aec52964da60 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998518, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.126@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:59.170037 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.163637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38707 (local address 127.0.0.1:36634)
0806 06:41:59.163779 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.163789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.163871 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.164011 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.164012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.164326 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:41:59.164367 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.164369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.164969 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.164970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.165515 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.165516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.165799 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.165800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.165836 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.166266 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:41:59.166271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:41:59.167616 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:41:59.169156 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:59.169161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:59.169163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:59.169334 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:59.169488 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:41:59.169489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:41:59.169489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:41:59.169538 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:41:59.169749 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:41:59.169756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:41:59.169807 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.169874 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.169970 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260806 06:41:59.170097 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.163702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.14.51.126:38707)
0806 06:41:59.163796 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.163798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.163839 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:41:59.163924 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.163924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.163932 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.163981 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:59.164446 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.164910 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.165579 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.165742 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.165788 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.167700 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:59.167702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:59.167702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:59.167708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:59.169093 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:59.169389 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:59.169390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:59.169390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:59.169437 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:59.169588 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:59.169589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:59.169589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:59.169695 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:59.169742 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.169902 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.170020 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":62}
I20260806 06:41:59.171878 22353 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:36634:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:41:59.172061 22353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:59.173455 22781 tablet_service.cc:1467] Processing CreateTablet for tablet d1a3e9ab1c01454080680bec85b4f848 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba3f4bdcb09f40c68a8ff38f0875ea1f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:59.173465 22493 tablet_service.cc:1467] Processing CreateTablet for tablet d1a3e9ab1c01454080680bec85b4f848 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba3f4bdcb09f40c68a8ff38f0875ea1f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:59.173619 22493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a3e9ab1c01454080680bec85b4f848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.173619 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a3e9ab1c01454080680bec85b4f848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.174749 22879 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: Bootstrap starting.
I20260806 06:41:59.174877 22636 tablet_service.cc:1467] Processing CreateTablet for tablet d1a3e9ab1c01454080680bec85b4f848 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba3f4bdcb09f40c68a8ff38f0875ea1f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:41:59.175029 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a3e9ab1c01454080680bec85b4f848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.175172 22879 tablet_bootstrap.cc:654] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.175737 22879 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: No bootstrap required, opened a new log
I20260806 06:41:59.175777 22879 ts_tablet_manager.cc:1397] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:59.175904 22879 raft_consensus.cc:348] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.175969 22879 raft_consensus.cc:374] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.175983 22879 raft_consensus.cc:729] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.176033 22879 consensus_queue.cc:260] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.176139 22879 ts_tablet_manager.cc:1428] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.176155 22897 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.176226 22897 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.176501 22880 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: Bootstrap starting.
I20260806 06:41:59.176540 22878 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: Bootstrap starting.
I20260806 06:41:59.176955 22880 tablet_bootstrap.cc:654] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.176954 22878 tablet_bootstrap.cc:654] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.177642 22878 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: No bootstrap required, opened a new log
I20260806 06:41:59.177699 22878 ts_tablet_manager.cc:1397] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:59.177858 22878 raft_consensus.cc:348] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.177939 22878 raft_consensus.cc:374] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.177973 22878 raft_consensus.cc:729] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef222ac6c1e41b38a180526dbcaa01a, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.178026 22878 consensus_queue.cc:260] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.178112 22878 ts_tablet_manager.cc:1428] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.178197 22884 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.178301 22884 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.178313 22880 tablet_bootstrap.cc:492] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: No bootstrap required, opened a new log
I20260806 06:41:59.178368 22880 ts_tablet_manager.cc:1397] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:41:59.178525 22880 raft_consensus.cc:348] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.178628 22880 raft_consensus.cc:374] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.178656 22880 raft_consensus.cc:729] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.178697 22880 consensus_queue.cc:260] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.178822 22880 ts_tablet_manager.cc:1428] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.178870 22885 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.178966 22885 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.214771 22885 raft_consensus.cc:482] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:59.214848 22885 raft_consensus.cc:504] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.214988 22885 leader_election.cc:288] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.215160 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a3e9ab1c01454080680bec85b4f848" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" is_pre_election: true
I20260806 06:41:59.215250 22513 raft_consensus.cc:2454] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.215196 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a3e9ab1c01454080680bec85b4f848" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327" is_pre_election: true
I20260806 06:41:59.215287 22801 raft_consensus.cc:2454] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.215412 22590 leader_election.cc:302] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef222ac6c1e41b38a180526dbcaa01a, 165da6ab00e84028ae02aec52964da60; no voters:
I20260806 06:41:59.215523 22885 raft_consensus.cc:2781] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:59.215565 22885 raft_consensus.cc:482] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:59.215580 22885 raft_consensus.cc:3037] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.216195 22885 raft_consensus.cc:504] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.216321 22885 leader_election.cc:288] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.216498 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a3e9ab1c01454080680bec85b4f848" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
I20260806 06:41:59.216498 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a3e9ab1c01454080680bec85b4f848" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327"
I20260806 06:41:59.216573 22801 raft_consensus.cc:3037] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.216574 22513 raft_consensus.cc:3037] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.217317 22801 raft_consensus.cc:2454] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.217317 22513 raft_consensus.cc:2454] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.217486 22587 leader_election.cc:302] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165da6ab00e84028ae02aec52964da60, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:59.217583 22885 raft_consensus.cc:2781] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:59.217638 22885 raft_consensus.cc:686] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [term 1 LEADER]: Becoming Leader. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Running, Role: LEADER
I20260806 06:41:59.217669 22885 consensus_queue.cc:237] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.217981 22912 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: TxnStatusTablet state changed. Reason: New leader 165da6ab00e84028ae02aec52964da60. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.218083 22912 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:59.218269 22924 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:41:59.218235 22352 catalog_manager.cc:5495] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 165da6ab00e84028ae02aec52964da60 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:59.218374 22924 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:41:59.225302 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.222936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.14.51.66:43387)
0806 06:41:59.222981 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.222983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.222992 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:41:59.223088 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.223089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.223105 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.223187 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.223297 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.223742 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.224494 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.224703 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.224772 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.224954 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.225062 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.225137 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.225191 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.225210 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:59.225320 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.222877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:43387 (local address 127.0.0.1:34942)
0806 06:41:59.222943 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.222954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.223003 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.223191 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.223192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.223194 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:59.223239 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.223241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.223832 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.223833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.224413 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.224415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.224785 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.224786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.224842 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.224865 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.225123 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.225133 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.225186 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.225234 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:41:59.226295 22513 raft_consensus.cc:1261] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.226537 22801 raft_consensus.cc:1261] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.226583 22912 consensus_queue.cc:1035] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.226770 22912 consensus_queue.cc:1035] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.227458 22884 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: TxnStatusTablet state changed. Reason: New leader 165da6ab00e84028ae02aec52964da60. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.227563 22884 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.227558 22897 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: TxnStatusTablet state changed. Reason: New leader 165da6ab00e84028ae02aec52964da60. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.227634 22897 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.228201 22912 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.228300 22912 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:59.228222 22885 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.228421 22885 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 165da6ab00e84028ae02aec52964da60: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:41:59.228389 22884 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.228456 22884 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 0ef222ac6c1e41b38a180526dbcaa01a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.229031 22891 mvcc.cc:205] Tried to move back new op lower bound from 7315449934749171712 to 7315449934715899904. Current Snapshot: MvccSnapshot[applied={T|T < 7315449934749171712 or (T in {7315449934749171712})}]
I20260806 06:41:59.229205 22897 tablet_replica.cc:439] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } } }
I20260806 06:41:59.229282 22897 tablet_replica.cc:442] T d1a3e9ab1c01454080680bec85b4f848 P 2e1907e616574d9cb83d9b98bff87327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:41:59.231796 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.229860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.14.51.66:43387)
0806 06:41:59.229894 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.229896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.229901 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:41:59.229917 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.229918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.229925 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.229967 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.230095 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.230483 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.231152 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.231335 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.231401 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.231558 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.231586 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.231646 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.231707 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.231728 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:59.236380 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.233836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:47573 (local address 127.14.51.66:43387)
0806 06:41:59.233871 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.233873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.234010 (+ 137us) server_negotiation.cc:407] Connection header received
0806 06:41:59.234067 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.234067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.234084 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.234126 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.234255 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.234662 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.235820 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.235983 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.236034 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.236202 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.236222 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.236258 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.236309 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.236325 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:59.236481 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.233781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:43387 (local address 127.14.51.66:47573)
0806 06:41:59.233929 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.233938 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.234011 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.234130 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.234130 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.234132 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:59.234198 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.234199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.234727 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.234728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.235746 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.235749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.236051 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.236052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.236119 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.236132 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.236259 (+ 127us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.236260 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.236307 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.236347 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260806 06:41:59.238456 22801 raft_consensus.cc:1261] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.238492 22513 raft_consensus.cc:1261] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.238684 22885 consensus_queue.cc:1035] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.238790 22929 consensus_queue.cc:1035] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.252509 22352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.252594 22352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.253165 22352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Sending DeleteTablet for 3 replicas of tablet 3eae7ad55340454f8578529735e6a472
I20260806 06:41:59.253409 22493 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eae7ad55340454f8578529735e6a472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58470
I20260806 06:41:59.253472 22781 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eae7ad55340454f8578529735e6a472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58344
I20260806 06:41:59.253501 22902 tablet_replica.cc:330] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: stopping tablet replica
I20260806 06:41:59.253502 22636 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eae7ad55340454f8578529735e6a472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34940
I20260806 06:41:59.253551 22902 raft_consensus.cc:2229] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.253556 22905 tablet_replica.cc:330] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: stopping tablet replica
I20260806 06:41:59.253584 22902 raft_consensus.cc:2258] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.253611 22905 raft_consensus.cc:2229] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.253645 22905 raft_consensus.cc:2258] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.253638 22903 tablet_replica.cc:330] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: stopping tablet replica
I20260806 06:41:59.253687 22903 raft_consensus.cc:2229] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:59.253772 22903 raft_consensus.cc:2258] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.253921 22902 ts_tablet_manager.cc:1905] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.254006 22905 ts_tablet_manager.cc:1905] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.254293 22903 ts_tablet_manager.cc:1905] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.255686 22905 ts_tablet_manager.cc:1918] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:59.255693 22903 ts_tablet_manager.cc:1918] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:59.255746 22905 log.cc:1196] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/3eae7ad55340454f8578529735e6a472
I20260806 06:41:59.255774 22903 log.cc:1196] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/3eae7ad55340454f8578529735e6a472
I20260806 06:41:59.255797 22902 ts_tablet_manager.cc:1918] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:59.255847 22902 log.cc:1196] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/3eae7ad55340454f8578529735e6a472
I20260806 06:41:59.255972 22905 ts_tablet_manager.cc:1939] T 3eae7ad55340454f8578529735e6a472 P 2e1907e616574d9cb83d9b98bff87327: Deleting consensus metadata
I20260806 06:41:59.256006 22903 ts_tablet_manager.cc:1939] T 3eae7ad55340454f8578529735e6a472 P 165da6ab00e84028ae02aec52964da60: Deleting consensus metadata
I20260806 06:41:59.256064 22902 ts_tablet_manager.cc:1939] T 3eae7ad55340454f8578529735e6a472 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting consensus metadata
I20260806 06:41:59.257489 22340 catalog_manager.cc:4841] TS 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627): tablet 3eae7ad55340454f8578529735e6a472 (table test-table [id=d8906ed1f5854bcab49eed3a69e5d020]) successfully deleted
I20260806 06:41:59.257520 22339 catalog_manager.cc:4841] TS 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387): tablet 3eae7ad55340454f8578529735e6a472 (table test-table [id=d8906ed1f5854bcab49eed3a69e5d020]) successfully deleted
I20260806 06:41:59.257766 22337 catalog_manager.cc:4841] TS 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353): tablet 3eae7ad55340454f8578529735e6a472 (table test-table [id=d8906ed1f5854bcab49eed3a69e5d020]) successfully deleted
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:41:59.278496 22405 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:41:59.272642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.14.51.126:38707)
0806 06:41:59.272720 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.272722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.272815 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:41:59.272866 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.272867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.272878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.272939 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:59.273517 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.274292 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.275004 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.275416 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.275491 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.278425 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36640: BlockingRecv error: recv got EOF from 127.0.0.1:36640 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM
I20260806 06:41:59.286276 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.279288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.14.51.126:38707)
0806 06:41:59.279324 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.279326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.279445 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:41:59.279505 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.279506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.279515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.279568 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:41:59.279989 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.280722 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.281351 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.281647 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.281709 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.283740 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:41:59.283742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:41:59.283742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:41:59.283749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:41:59.285258 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:59.285573 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:59.285574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:59.285575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:59.285635 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:41:59.285798 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:41:59.285799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:41:59.285799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:41:59.285924 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:41:59.286014 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.286103 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.286181 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:41:59.287752 22352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:59.287920 22352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:59.289228 22636 tablet_service.cc:1467] Processing CreateTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 (DEFAULT_TABLE table=test-table [id=92380fd16cce4be68019608bc76dfb43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.289254 22781 tablet_service.cc:1467] Processing CreateTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 (DEFAULT_TABLE table=test-table [id=92380fd16cce4be68019608bc76dfb43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.289260 22493 tablet_service.cc:1467] Processing CreateTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 (DEFAULT_TABLE table=test-table [id=92380fd16cce4be68019608bc76dfb43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.289369 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3e3c3f0aa934f709ac755957367d4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.289371 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3e3c3f0aa934f709ac755957367d4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.289388 22493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3e3c3f0aa934f709ac755957367d4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.290653 22878 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Bootstrap starting.
I20260806 06:41:59.290664 22879 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Bootstrap starting.
I20260806 06:41:59.290936 22879 tablet_bootstrap.cc:654] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.291047 22878 tablet_bootstrap.cc:654] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.291410 22880 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Bootstrap starting.
I20260806 06:41:59.291755 22880 tablet_bootstrap.cc:654] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.291814 22879 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: No bootstrap required, opened a new log
I20260806 06:41:59.291862 22879 ts_tablet_manager.cc:1397] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:41:59.291994 22879 raft_consensus.cc:348] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.292083 22879 raft_consensus.cc:374] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.292112 22879 raft_consensus.cc:729] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.292163 22879 consensus_queue.cc:260] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.292282 22879 ts_tablet_manager.cc:1428] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:59.292558 22880 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: No bootstrap required, opened a new log
I20260806 06:41:59.292604 22880 ts_tablet_manager.cc:1397] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:59.292721 22880 raft_consensus.cc:348] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.292788 22880 raft_consensus.cc:374] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.292831 22880 raft_consensus.cc:729] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.292869 22880 consensus_queue.cc:260] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.292953 22880 ts_tablet_manager.cc:1428] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.293156 22878 tablet_bootstrap.cc:492] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: No bootstrap required, opened a new log
I20260806 06:41:59.293206 22878 ts_tablet_manager.cc:1397] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:41:59.293361 22878 raft_consensus.cc:348] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.293432 22878 raft_consensus.cc:374] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.293452 22878 raft_consensus.cc:729] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef222ac6c1e41b38a180526dbcaa01a, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.293490 22878 consensus_queue.cc:260] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.293565 22878 ts_tablet_manager.cc:1428] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.326843 22912 raft_consensus.cc:482] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:59.326938 22912 raft_consensus.cc:504] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.327119 22912 leader_election.cc:288] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.327389 22512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3e3c3f0aa934f709ac755957367d4b5" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" is_pre_election: true
I20260806 06:41:59.327395 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3e3c3f0aa934f709ac755957367d4b5" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327" is_pre_election: true
I20260806 06:41:59.327472 22512 raft_consensus.cc:2454] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.327490 22801 raft_consensus.cc:2454] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 0.
I20260806 06:41:59.327662 22590 leader_election.cc:302] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef222ac6c1e41b38a180526dbcaa01a, 165da6ab00e84028ae02aec52964da60; no voters:
I20260806 06:41:59.327806 22912 raft_consensus.cc:2781] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:59.327838 22912 raft_consensus.cc:482] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:59.327857 22912 raft_consensus.cc:3037] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.328737 22912 raft_consensus.cc:504] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.328868 22912 leader_election.cc:288] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353)
I20260806 06:41:59.329001 22512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3e3c3f0aa934f709ac755957367d4b5" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
I20260806 06:41:59.329089 22512 raft_consensus.cc:3037] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.329099 22801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3e3c3f0aa934f709ac755957367d4b5" candidate_uuid: "165da6ab00e84028ae02aec52964da60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1907e616574d9cb83d9b98bff87327"
I20260806 06:41:59.329155 22801 raft_consensus.cc:3037] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.329958 22801 raft_consensus.cc:2454] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.330139 22587 leader_election.cc:302] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165da6ab00e84028ae02aec52964da60, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:59.330227 22912 raft_consensus.cc:2781] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:59.330274 22912 raft_consensus.cc:686] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 1 LEADER]: Becoming Leader. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Running, Role: LEADER
I20260806 06:41:59.330327 22912 consensus_queue.cc:237] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.330391 22512 raft_consensus.cc:2454] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 165da6ab00e84028ae02aec52964da60 in term 1.
I20260806 06:41:59.330854 22352 catalog_manager.cc:5495] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 165da6ab00e84028ae02aec52964da60 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "165da6ab00e84028ae02aec52964da60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } health_report { overall_health: UNKNOWN } } }
I20260806 06:41:59.339332 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.336881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.14.51.66:43387)
0806 06:41:59.336923 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.336926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.336955 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:41:59.337024 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.337025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.337035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.337114 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.337288 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.337755 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.338655 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.338848 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.338918 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.339066 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.339097 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.339179 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.339228 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.339264 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:41:59.339986 22512 raft_consensus.cc:1261] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.339986 22801 raft_consensus.cc:1261] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Refusing update from remote peer 165da6ab00e84028ae02aec52964da60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.340195 22885 consensus_queue.cc:1035] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.340273 22912 consensus_queue.cc:1035] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.343216 22352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.343307 22352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.343910 22352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Sending DeleteTablet for 3 replicas of tablet f3e3c3f0aa934f709ac755957367d4b5
I20260806 06:41:59.344195 22636 tablet_service.cc:1514] Processing DeleteTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34940
I20260806 06:41:59.344195 22781 tablet_service.cc:1514] Processing DeleteTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58344
I20260806 06:41:59.344286 22903 tablet_replica.cc:330] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: stopping tablet replica
I20260806 06:41:59.344285 22905 tablet_replica.cc:330] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: stopping tablet replica
I20260806 06:41:59.344336 22903 raft_consensus.cc:2229] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:59.344336 22905 raft_consensus.cc:2229] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.344378 22905 raft_consensus.cc:2258] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.344389 22903 raft_consensus.cc:2258] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.344646 22905 ts_tablet_manager.cc:1905] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.344666 22493 tablet_service.cc:1514] Processing DeleteTablet for tablet f3e3c3f0aa934f709ac755957367d4b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58470
I20260806 06:41:59.344740 22902 tablet_replica.cc:330] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: stopping tablet replica
I20260806 06:41:59.344794 22902 raft_consensus.cc:2229] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.344842 22902 raft_consensus.cc:2258] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.344996 22903 ts_tablet_manager.cc:1905] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.346172 22905 ts_tablet_manager.cc:1918] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:59.346235 22905 log.cc:1196] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/f3e3c3f0aa934f709ac755957367d4b5
I20260806 06:41:59.346446 22905 ts_tablet_manager.cc:1939] T f3e3c3f0aa934f709ac755957367d4b5 P 2e1907e616574d9cb83d9b98bff87327: Deleting consensus metadata
I20260806 06:41:59.346529 22903 ts_tablet_manager.cc:1918] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:59.346611 22903 log.cc:1196] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/f3e3c3f0aa934f709ac755957367d4b5
I20260806 06:41:59.346781 22337 catalog_manager.cc:4841] TS 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353): tablet f3e3c3f0aa934f709ac755957367d4b5 (table test-table [id=92380fd16cce4be68019608bc76dfb43]) successfully deleted
I20260806 06:41:59.346830 22903 ts_tablet_manager.cc:1939] T f3e3c3f0aa934f709ac755957367d4b5 P 165da6ab00e84028ae02aec52964da60: Deleting consensus metadata
I20260806 06:41:59.347105 22339 catalog_manager.cc:4841] TS 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387): tablet f3e3c3f0aa934f709ac755957367d4b5 (table test-table [id=92380fd16cce4be68019608bc76dfb43]) successfully deleted
I20260806 06:41:59.347337 22902 ts_tablet_manager.cc:1905] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.347368 22352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:41:59.347514 22352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:41:59.348671 22493 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0520956b84eda88252a49c3163715 (DEFAULT_TABLE table=test-table [id=030a68d77332467db502a6be4eb44181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.348713 22781 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0520956b84eda88252a49c3163715 (DEFAULT_TABLE table=test-table [id=030a68d77332467db502a6be4eb44181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.348778 22493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0520956b84eda88252a49c3163715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.348824 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0520956b84eda88252a49c3163715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.348901 22902 ts_tablet_manager.cc:1918] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:41:59.348948 22902 log.cc:1196] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/f3e3c3f0aa934f709ac755957367d4b5
I20260806 06:41:59.349167 22902 ts_tablet_manager.cc:1939] T f3e3c3f0aa934f709ac755957367d4b5 P 0ef222ac6c1e41b38a180526dbcaa01a: Deleting consensus metadata
I20260806 06:41:59.349987 22878 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: Bootstrap starting.
I20260806 06:41:59.350054 22879 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Bootstrap starting.
I20260806 06:41:59.350144 22340 catalog_manager.cc:4841] TS 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627): tablet f3e3c3f0aa934f709ac755957367d4b5 (table test-table [id=92380fd16cce4be68019608bc76dfb43]) successfully deleted
I20260806 06:41:59.350351 22879 tablet_bootstrap.cc:654] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.350375 22878 tablet_bootstrap.cc:654] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.350904 22636 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0520956b84eda88252a49c3163715 (DEFAULT_TABLE table=test-table [id=030a68d77332467db502a6be4eb44181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:41:59.351008 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0520956b84eda88252a49c3163715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:41:59.351317 22879 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: No bootstrap required, opened a new log
I20260806 06:41:59.351368 22879 ts_tablet_manager.cc:1397] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:41:59.351462 22879 raft_consensus.cc:348] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.351537 22879 raft_consensus.cc:374] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.351558 22879 raft_consensus.cc:729] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.351616 22879 consensus_queue.cc:260] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.351732 22879 ts_tablet_manager.cc:1428] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:41:59.352252 22880 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Bootstrap starting.
I20260806 06:41:59.352496 22878 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: No bootstrap required, opened a new log
I20260806 06:41:59.352560 22878 ts_tablet_manager.cc:1397] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:41:59.352669 22880 tablet_bootstrap.cc:654] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Neither blocks nor log segments found. Creating new log.
I20260806 06:41:59.352710 22878 raft_consensus.cc:348] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.352790 22878 raft_consensus.cc:374] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.352820 22878 raft_consensus.cc:729] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef222ac6c1e41b38a180526dbcaa01a, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.352860 22878 consensus_queue.cc:260] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.352958 22878 ts_tablet_manager.cc:1428] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:41:59.353416 22880 tablet_bootstrap.cc:492] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: No bootstrap required, opened a new log
I20260806 06:41:59.353474 22880 ts_tablet_manager.cc:1397] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:41:59.353608 22880 raft_consensus.cc:348] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.353672 22880 raft_consensus.cc:374] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:41:59.353691 22880 raft_consensus.cc:729] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165da6ab00e84028ae02aec52964da60, State: Initialized, Role: FOLLOWER
I20260806 06:41:59.353739 22880 consensus_queue.cc:260] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.353821 22880 ts_tablet_manager.cc:1428] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:41:59.361025 22897 raft_consensus.cc:482] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:41:59.361099 22897 raft_consensus.cc:504] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.361243 22897 leader_election.cc:288] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387)
I20260806 06:41:59.361403 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa0520956b84eda88252a49c3163715" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165da6ab00e84028ae02aec52964da60" is_pre_election: true
I20260806 06:41:59.361420 22512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa0520956b84eda88252a49c3163715" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" is_pre_election: true
I20260806 06:41:59.361493 22512 raft_consensus.cc:2454] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 0.
I20260806 06:41:59.361497 22656 raft_consensus.cc:2454] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 0.
I20260806 06:41:59.361646 22732 leader_election.cc:302] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef222ac6c1e41b38a180526dbcaa01a, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:59.361752 22897 raft_consensus.cc:2781] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:41:59.361783 22897 raft_consensus.cc:482] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:41:59.361801 22897 raft_consensus.cc:3037] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.362591 22897 raft_consensus.cc:504] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.362720 22897 leader_election.cc:288] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627), 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387)
I20260806 06:41:59.362902 22512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa0520956b84eda88252a49c3163715" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef222ac6c1e41b38a180526dbcaa01a"
I20260806 06:41:59.362927 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa0520956b84eda88252a49c3163715" candidate_uuid: "2e1907e616574d9cb83d9b98bff87327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165da6ab00e84028ae02aec52964da60"
I20260806 06:41:59.362978 22512 raft_consensus.cc:3037] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.362988 22656 raft_consensus.cc:3037] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:41:59.363560 22656 raft_consensus.cc:2454] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 1.
I20260806 06:41:59.363704 22512 raft_consensus.cc:2454] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e1907e616574d9cb83d9b98bff87327 in term 1.
I20260806 06:41:59.363742 22731 leader_election.cc:302] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165da6ab00e84028ae02aec52964da60, 2e1907e616574d9cb83d9b98bff87327; no voters:
I20260806 06:41:59.363837 22897 raft_consensus.cc:2781] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:41:59.363878 22897 raft_consensus.cc:686] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 1 LEADER]: Becoming Leader. State: Replica: 2e1907e616574d9cb83d9b98bff87327, State: Running, Role: LEADER
I20260806 06:41:59.363930 22897 consensus_queue.cc:237] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } }
I20260806 06:41:59.364455 22352 catalog_manager.cc:5495] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67). New cstate: current_term: 1 leader_uuid: "2e1907e616574d9cb83d9b98bff87327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e1907e616574d9cb83d9b98bff87327" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 36353 } health_report { overall_health: HEALTHY } } }
I20260806 06:41:59.373961 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.371516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.14.51.67:36353)
0806 06:41:59.371562 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.371564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.371589 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:41:59.371659 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.371660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.371668 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.371709 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.371825 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.372284 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.373412 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.373586 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.373646 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.373780 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.373803 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.373839 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.373898 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.373913 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:41:59.378854 22961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.376178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:43387 (local address 127.14.51.67:46817)
0806 06:41:59.376339 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.376350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.376407 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.376558 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.376558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.376560 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:59.376606 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.376607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.377448 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.377449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.378105 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.378107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.378528 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.378529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.378574 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.378587 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.378740 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.378741 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.378771 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.378802 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":102,"threads_started":1}
I20260806 06:41:59.378857 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.376217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46817 (local address 127.14.51.66:43387)
0806 06:41:59.376278 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.376280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.376411 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:41:59.376445 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.376446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.376458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.376532 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.376658 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.377383 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.378170 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.378470 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.378526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.378658 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.378687 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.378740 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.378773 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.378788 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:41:59.380661 22656 raft_consensus.cc:1261] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Refusing update from remote peer 2e1907e616574d9cb83d9b98bff87327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.380671 22512 raft_consensus.cc:1261] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Refusing update from remote peer 2e1907e616574d9cb83d9b98bff87327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:41:59.380848 22897 consensus_queue.cc:1035] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "165da6ab00e84028ae02aec52964da60" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 43387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.380944 22886 consensus_queue.cc:1035] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef222ac6c1e41b38a180526dbcaa01a" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:41:59.387483 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.385273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:36353 (local address 127.14.51.66:54375)
0806 06:41:59.385365 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.385375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:41:59.385410 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:41:59.385580 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:41:59.385581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:41:59.385582 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:41:59.385611 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.385612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.386171 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.386173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.386685 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:41:59.386687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.387034 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.387035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:41:59.387087 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.387093 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.387289 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.387292 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:41:59.387354 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:41:59.387413 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 06:41:59.387516 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.385362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:54375 (local address 127.14.51.67:36353)
0806 06:41:59.385399 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:41:59.385401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:41:59.385409 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:41:59.385461 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:41:59.385462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:41:59.385472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:41:59.385521 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:41:59.385683 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.386099 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.386763 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:41:59.386956 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:41:59.387031 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:41:59.387207 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:41:59.387235 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:41:59.387287 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:41:59.387425 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:41:59.387448 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:41:59.393496 22352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.393599 22352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:41:59.394173 22352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a5a3805f9844439862437760cf8315: Sending DeleteTablet for 3 replicas of tablet 7fa0520956b84eda88252a49c3163715
I20260806 06:41:59.394393 22636 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa0520956b84eda88252a49c3163715 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:34940
I20260806 06:41:59.394410 22781 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa0520956b84eda88252a49c3163715 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58344
I20260806 06:41:59.394441 22493 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa0520956b84eda88252a49c3163715 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:41:59 UTC) from {username='oryx', principal='oryx/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:58470
I20260806 06:41:59.394475 22903 tablet_replica.cc:330] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: stopping tablet replica
I20260806 06:41:59.394487 22905 tablet_replica.cc:330] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: stopping tablet replica
I20260806 06:41:59.394510 22902 tablet_replica.cc:330] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a: stopping tablet replica
I20260806 06:41:59.394520 22903 raft_consensus.cc:2229] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.394531 22905 raft_consensus.cc:2229] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:41:59.394549 22903 raft_consensus.cc:2258] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.394562 22902 raft_consensus.cc:2229] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:41:59.394573 22905 raft_consensus.cc:2258] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.394591 22902 raft_consensus.cc:2258] T 7fa0520956b84eda88252a49c3163715 P 0ef222ac6c1e41b38a180526dbcaa01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:41:59.394898 22903 ts_tablet_manager.cc:1905] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.395239 22905 ts_tablet_manager.cc:1905] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:41:59.395242 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 22427
I20260806 06:41:59.398281 22903 ts_tablet_manager.cc:1918] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:59.398322 22903 log.cc:1196] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/7fa0520956b84eda88252a49c3163715
I20260806 06:41:59.398478 22903 ts_tablet_manager.cc:1939] T 7fa0520956b84eda88252a49c3163715 P 165da6ab00e84028ae02aec52964da60: Deleting consensus metadata
I20260806 06:41:59.398662 22905 ts_tablet_manager.cc:1918] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:41:59.398722 22905 log.cc:1196] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/7fa0520956b84eda88252a49c3163715
I20260806 06:41:59.398756 22339 catalog_manager.cc:4841] TS 165da6ab00e84028ae02aec52964da60 (127.14.51.66:43387): tablet 7fa0520956b84eda88252a49c3163715 (table test-table [id=030a68d77332467db502a6be4eb44181]) successfully deleted
I20260806 06:41:59.398941 22905 ts_tablet_manager.cc:1939] T 7fa0520956b84eda88252a49c3163715 P 2e1907e616574d9cb83d9b98bff87327: Deleting consensus metadata
I20260806 06:41:59.399258 22337 catalog_manager.cc:4841] TS 2e1907e616574d9cb83d9b98bff87327 (127.14.51.67:36353): tablet 7fa0520956b84eda88252a49c3163715 (table test-table [id=030a68d77332467db502a6be4eb44181]) successfully deleted
W20260806 06:41:59.402520 22340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:38627 (error 108)
I20260806 06:41:59.403064 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:41:59.402869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:38627 (local address 127.0.0.1:52946)
0806 06:41:59.402975 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:41:59.403020 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260806 06:41:59.403185 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 22570
W20260806 06:41:59.407069 22340 catalog_manager.cc:4584] TS 0ef222ac6c1e41b38a180526dbcaa01a (127.14.51.65:38627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7fa0520956b84eda88252a49c3163715: Network error: Client connection negotiation failed: client connection to 127.14.51.65:38627: connect: Connection refused (error 111)
I20260806 06:41:59.411458 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 22712
I20260806 06:41:59.417686 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 22321
2026-08-06T06:41:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2595 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22968](info): set up 2 sockets
Aug 06 06:41:59 dist-test-slave-1xrh krb5kdc[22968](info): commencing operation
krb5kdc: starting...
W20260806 06:42:00.531314 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:42:00 dist-test-slave-1xrh krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:42:00.645946 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:42297
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:42297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:00.727047 22982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:00.727347 22982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:00.727389 22982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:00.728727 22982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:00.728798 22982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:00.728838 22982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:00.728869 22982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:00.728901 22982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:00.730506 22982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:42297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:42297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:00.730785 22982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:00.731031 22982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:00.736018 22987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:00.736027 22990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:00.736030 22988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:00.736301 22982 server_base.cc:1034] running on GCE node
I20260806 06:42:00.736621 22982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:00.736953 22982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:00.738142 22982 hybrid_clock.cc:648] HybridClock initialized: now 1785998520738119 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T06:42:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1323 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:42:00 dist-test-slave-1xrh krb5kdc[22995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:42:00 dist-test-slave-1xrh krb5kdc[22995](info): set up 2 sockets
Aug 06 06:42:00 dist-test-slave-1xrh krb5kdc[22995](info): commencing operation
krb5kdc: starting...
W20260806 06:42:01.852372 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:42:01 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:42:01.972301 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38307
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:02.052584 23008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.052803 23008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.052839 23008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.053984 23008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:02.054047 23008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.054073 23008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.054095 23008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:02.054122 23008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:02.055552 23008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.055817 23008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.056033 23008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:02.060988 23016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.060992 23014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.061007 23008 server_base.cc:1034] running on GCE node
W20260806 06:42:02.060984 23013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.061460 23008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.061729 23008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.062882 23008 hybrid_clock.cc:648] HybridClock initialized: now 1785998522062867 us; error 42 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.070276 23008 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:42:02.070640 23008 webserver.cc:460] Webserver started at http://127.14.51.126:45953/ using document root <none> and password file <none>
I20260806 06:42:02.070809 23008 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.070848 23008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.070940 23008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:02.071928 23008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "dbf1bda79f514f7e82bbdc377a0abca8"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "c6db29c56aa886ac39c65ab8e02b3eb9"
server_key_iv: "d3cd48ebc71b2a1da18752baed7388d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.072214 23008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "dbf1bda79f514f7e82bbdc377a0abca8"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "c6db29c56aa886ac39c65ab8e02b3eb9"
server_key_iv: "d3cd48ebc71b2a1da18752baed7388d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.073398 23008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:42:02.074030 23023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.074182 23008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:02.074267 23008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "dbf1bda79f514f7e82bbdc377a0abca8"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "c6db29c56aa886ac39c65ab8e02b3eb9"
server_key_iv: "d3cd48ebc71b2a1da18752baed7388d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.074321 23008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.089831 23008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.091686 23008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.091789 23008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.095484 23008 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38307
I20260806 06:42:02.095533 23077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38307 every 8 connection(s)
I20260806 06:42:02.095844 23008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:42:02.096191 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23008
I20260806 06:42:02.096319 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:42:02.096513 14541 external_mini_cluster.cc:1428] Setting key ecf103ef4082ac8613ec7092ca011493
I20260806 06:42:02.096877 23078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:02.100093 23078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Bootstrap starting.
I20260806 06:42:02.100701 23078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:02.100991 23078 log.cc:824] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:02.101749 23078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: No bootstrap required, opened a new log
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:02.103659 23078 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.103917 23078 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:02.103962 23078 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf1bda79f514f7e82bbdc377a0abca8, State: Initialized, Role: FOLLOWER
I20260806 06:42:02.104116 23078 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.104214 23078 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:42:02.104259 23078 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:42:02.104306 23078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:02.104811 23078 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.104924 23078 leader_election.cc:302] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf1bda79f514f7e82bbdc377a0abca8; no voters:
I20260806 06:42:02.105100 23078 leader_election.cc:288] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:42:02.105129 23083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:42:02.105233 23083 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 LEADER]: Becoming Leader. State: Replica: dbf1bda79f514f7e82bbdc377a0abca8, State: Running, Role: LEADER
I20260806 06:42:02.105312 23083 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.105348 23078 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:42:02.105623 23083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } } }
I20260806 06:42:02.105695 23083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:02.105824 23083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf1bda79f514f7e82bbdc377a0abca8. Latest consensus state: current_term: 1 leader_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } } }
I20260806 06:42:02.105981 23083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:02.106153 23088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:42:02.106683 23088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:42:02.107950 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.097033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.14.51.126:38307)
0806 06:42:02.097191 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.097200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.097229 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:42:02.097756 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.097764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.097822 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.097981 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.098555 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.099949 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.100723 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.101304 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.101385 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.103681 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:02.103685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:02.103685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:02.103695 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:02.105848 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.106288 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.106291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.106291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.106400 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.106609 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.106610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.106610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.106919 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:02.107082 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:02.107610 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 06:42:02.107751 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260806 06:42:02.108285 23088 catalog_manager.cc:1347] Generated new cluster ID: af938e45e9bf4d239f3d963cdf771277
I20260806 06:42:02.108340 23088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:42:02.121644 23088 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:42:02.122159 23088 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:42:02.126278 23088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Generated new TSK 0
I20260806 06:42:02.126441 23088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:42:02.158104 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:02.251066 23104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.251317 23104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.251338 23104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.252540 23104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.252586 23104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.252614 23104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:42:02.254007 23104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.254225 23104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.254446 23104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:02.259693 23112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.259693 23110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.259707 23109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.259819 23104 server_base.cc:1034] running on GCE node
I20260806 06:42:02.260182 23104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.260442 23104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.261596 23104 hybrid_clock.cc:648] HybridClock initialized: now 1785998522261590 us; error 34 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.264145 23104 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:42:02.264590 23104 webserver.cc:460] Webserver started at http://127.14.51.65:36135/ using document root <none> and password file <none>
I20260806 06:42:02.264816 23104 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.264876 23104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.265014 23104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:02.265973 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "f7b4404788a94e9d8088eade5b5257cf"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "defc79955950d7d189b1762e79b9229d"
server_key_iv: "c5869fa933fa13e9497ae39795d79aa4"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.266319 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "f7b4404788a94e9d8088eade5b5257cf"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "defc79955950d7d189b1762e79b9229d"
server_key_iv: "c5869fa933fa13e9497ae39795d79aa4"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.267707 23104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:42:02.268388 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.268550 23104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.268648 23104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "f7b4404788a94e9d8088eade5b5257cf"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "defc79955950d7d189b1762e79b9229d"
server_key_iv: "c5869fa933fa13e9497ae39795d79aa4"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.268718 23104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.278170 23104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.280252 23104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.280390 23104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.280581 23104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:02.280840 23104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:02.280908 23104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.280947 23104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:02.280982 23104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.286682 23104 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34641
I20260806 06:42:02.286731 23237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34641 every 8 connection(s)
I20260806 06:42:02.287001 23104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:02.292709 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23104
I20260806 06:42:02.292834 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:42:02.293011 14541 external_mini_cluster.cc:1428] Setting key f4d653bf737afdfba39b5c04539308b7
I20260806 06:42:02.296048 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.287649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:36645 (local address 127.14.51.126:38307)
0806 06:42:02.287726 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.287729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.287872 (+ 143us) server_negotiation.cc:407] Connection header received
0806 06:42:02.288294 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.288295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.288304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.288352 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.289189 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.289705 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.290615 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.290801 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.290848 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.292853 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:02.292856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:02.292856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:02.292863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:02.294623 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.295017 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.295018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.295018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.295087 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.295395 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.295397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.295397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.295527 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:02.295604 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:02.295861 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 06:42:02.295956 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:42:02.296258 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.287616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:36645)
0806 06:42:02.287766 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.287798 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.288218 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.288407 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.288411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.288953 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.289121 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.289125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.289776 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.289777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.290531 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.290532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.290867 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.290867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.290966 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.291393 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.291401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.292759 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:02.294714 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.294720 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.294722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.294948 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.295196 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.295197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.295197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.295326 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.295595 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:02.295601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:02.295732 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:42:02.295828 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:42:02.295989 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260806 06:42:02.296758 23238 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38307
I20260806 06:42:02.296834 23238 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:02.297015 23238 heartbeater.cc:511] Master 127.14.51.126:38307 requested a full tablet report, sending...
I20260806 06:42:02.297533 23040 ts_manager.cc:194] Registered new tserver with Master: f7b4404788a94e9d8088eade5b5257cf (127.14.51.65:34641)
I20260806 06:42:02.298507 23040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:36645
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:42:02.317528 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:02.396523 23244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.396714 23244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.396737 23244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.397835 23244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.397868 23244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.397894 23244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:42:02.399246 23244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.399410 23244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.399569 23244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:02.404335 23250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.404335 23252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.404325 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.404475 23244 server_base.cc:1034] running on GCE node
I20260806 06:42:02.404811 23244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.405071 23244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.406199 23244 hybrid_clock.cc:648] HybridClock initialized: now 1785998522406186 us; error 29 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.408660 23244 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:42:02.409057 23244 webserver.cc:460] Webserver started at http://127.14.51.66:45305/ using document root <none> and password file <none>
I20260806 06:42:02.409289 23244 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.409351 23244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.409494 23244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:02.410715 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "007475be36aa420ebe874ea553a25131"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "ca52a536600837992fabac6d3021c65b"
server_key_iv: "d1e5cbb7d1b1d02b2de12e7e7de7f5d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.411182 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "007475be36aa420ebe874ea553a25131"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "ca52a536600837992fabac6d3021c65b"
server_key_iv: "d1e5cbb7d1b1d02b2de12e7e7de7f5d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.412501 23244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:42:02.413177 23259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.413383 23244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.413483 23244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "007475be36aa420ebe874ea553a25131"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "ca52a536600837992fabac6d3021c65b"
server_key_iv: "d1e5cbb7d1b1d02b2de12e7e7de7f5d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.413569 23244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.424427 23244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.426577 23244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.426741 23244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.426945 23244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:02.427294 23244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:02.427350 23244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.427384 23244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:02.427419 23244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.433157 23244 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:44741
I20260806 06:42:02.433240 23376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:44741 every 8 connection(s)
I20260806 06:42:02.433521 23244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:02.441506 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.434152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:42001 (local address 127.14.51.126:38307)
0806 06:42:02.434214 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.434216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.434388 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:42:02.434806 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.434807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.434816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.434872 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.435776 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.436238 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.437236 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.437412 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.437460 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.439430 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:02.439432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:02.439432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:02.439439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:02.440522 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.440838 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.440839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.440839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.440883 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.441064 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.441064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.441065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.441153 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:02.441198 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:02.441372 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:42:02.441432 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:42:02.441668 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.434115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:42001)
0806 06:42:02.434265 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.434302 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.434730 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.434925 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.434927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.435534 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.435702 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.435707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.436318 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.436318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.437163 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.437165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.437478 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.437478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.437574 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.437952 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.437959 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.439335 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:02.440584 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.440589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.440590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.440782 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.440934 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.440934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.440935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.441018 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.441199 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:02.441202 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:02.441293 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 06:42:02.441364 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:42:02.441462 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260806 06:42:02.441707 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23244
I20260806 06:42:02.441826 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:42:02.442051 14541 external_mini_cluster.cc:1428] Setting key e0788f1c4a221db3058186471a0bec71
I20260806 06:42:02.442171 23377 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38307
I20260806 06:42:02.442270 23377 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:02.442435 23377 heartbeater.cc:511] Master 127.14.51.126:38307 requested a full tablet report, sending...
I20260806 06:42:02.442783 23040 ts_manager.cc:194] Registered new tserver with Master: 007475be36aa420ebe874ea553a25131 (127.14.51.66:44741)
I20260806 06:42:02.443334 23040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:42001
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:42:02.466408 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:02.544494 23382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.544672 23382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.544694 23382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.545765 23382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.545804 23382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.545830 23382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:42:02.547176 23382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.547343 23382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.547511 23382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:42:02.552284 23382 server_base.cc:1034] running on GCE node
W20260806 06:42:02.552349 23390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.552250 23387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.552259 23388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.552685 23382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.552909 23382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.554052 23382 hybrid_clock.cc:648] HybridClock initialized: now 1785998522554026 us; error 40 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.556377 23382 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:42:02.556803 23382 webserver.cc:460] Webserver started at http://127.14.51.67:36071/ using document root <none> and password file <none>
I20260806 06:42:02.557035 23382 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.557099 23382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.557230 23382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:02.558203 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "6734e9ce946e47f4855027f3d8937b2c"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "a8f7caddaccbbbfa9e40fee39f391782"
server_key_iv: "c4c81f766819440a695dc86644243047"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.558544 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "6734e9ce946e47f4855027f3d8937b2c"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "a8f7caddaccbbbfa9e40fee39f391782"
server_key_iv: "c4c81f766819440a695dc86644243047"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.559841 23382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:42:02.560501 23398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.560660 23382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:02.560748 23382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "6734e9ce946e47f4855027f3d8937b2c"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "a8f7caddaccbbbfa9e40fee39f391782"
server_key_iv: "c4c81f766819440a695dc86644243047"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.560817 23382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.570622 23382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.572315 23382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.572458 23382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.572666 23382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:02.572942 23382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:02.572990 23382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.573028 23382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:02.573057 23382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.578923 23382 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:43347
I20260806 06:42:02.578958 23514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:43347 every 8 connection(s)
I20260806 06:42:02.579329 23382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:42:02.580456 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23382
I20260806 06:42:02.580581 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:42:02.580760 14541 external_mini_cluster.cc:1428] Setting key 82dde0f786e191d0b46ad4c9b5133da8
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:02.587939 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.579972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:36271 (local address 127.14.51.126:38307)
0806 06:42:02.580016 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.580018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.580186 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:42:02.580725 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.580727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.580739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.580787 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.581626 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.582133 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.583152 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.583347 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.583434 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.585428 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:02.585430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:02.585430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:02.585438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:02.586591 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.586984 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.586986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.586986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.587049 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:02.587319 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:02.587320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:02.587320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:02.587429 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:02.587504 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:02.587769 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:42:02.587850 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:42:02.588093 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.579942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:36271)
0806 06:42:02.580070 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.580110 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.580634 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.580856 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.580861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.581364 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.581527 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.581531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.582233 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.582234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.583042 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.583045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.583447 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.583448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.583573 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.584060 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.584068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.585336 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:02.586673 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.586679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.586682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.586900 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.587145 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:02.587146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:02.587146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:02.587245 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:02.587495 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:02.587501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:02.587633 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:42:02.587733 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:42:02.587877 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260806 06:42:02.588575 23515 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38307
I20260806 06:42:02.588651 23515 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:02.588773 23515 heartbeater.cc:511] Master 127.14.51.126:38307 requested a full tablet report, sending...
I20260806 06:42:02.589102 23040 ts_manager.cc:194] Registered new tserver with Master: 6734e9ce946e47f4855027f3d8937b2c (127.14.51.67:43347)
I20260806 06:42:02.589457 23040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:36271
I20260806 06:42:02.592190 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
Entry for principal oryx/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab.
WARNING: no policy specified for oryx/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
Entry for principal oryx/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab.
WARNING: no policy specified for oryx/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
Entry for principal oryx/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab.
I20260806 06:42:02.661454 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23104
I20260806 06:42:02.666950 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23244
I20260806 06:42:02.672341 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23382
I20260806 06:42:02.676779 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23008
I20260806 06:42:02.682704 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38307
--webserver_interface=127.14.51.126
--webserver_port=45953
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:02.770177 23524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.770385 23524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.770409 23524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.771697 23524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:02.771737 23524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.771754 23524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.771766 23524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:02.771780 23524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:02.773367 23524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.773581 23524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.773810 23524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:02.778829 23530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.778956 23524 server_base.cc:1034] running on GCE node
W20260806 06:42:02.778863 23529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.779031 23532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.779318 23524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.779627 23524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.780803 23524 hybrid_clock.cc:648] HybridClock initialized: now 1785998522780786 us; error 30 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.783390 23524 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:42:02.783872 23524 webserver.cc:460] Webserver started at http://127.14.51.126:45953/ using document root <none> and password file <none>
I20260806 06:42:02.784139 23524 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.784231 23524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.785595 23524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:42:02.786207 23541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.786379 23524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:02.786481 23524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "dbf1bda79f514f7e82bbdc377a0abca8"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "c6db29c56aa886ac39c65ab8e02b3eb9"
server_key_iv: "d3cd48ebc71b2a1da18752baed7388d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.786803 23524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.803992 23524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.806352 23524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.806514 23524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.810684 23524 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38307
I20260806 06:42:02.810714 23594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38307 every 8 connection(s)
I20260806 06:42:02.811084 23524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:42:02.812492 23595 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:42:02.813066 23595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Bootstrap starting.
I20260806 06:42:02.817072 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23524
I20260806 06:42:02.817340 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:34641
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:38307
--webserver_port=36135
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:42:02.820984 23595 log.cc:824] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:02.822623 23595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:42:02.822919 23595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Bootstrap complete.
I20260806 06:42:02.825084 23595 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.825313 23595 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf1bda79f514f7e82bbdc377a0abca8, State: Initialized, Role: FOLLOWER
I20260806 06:42:02.825497 23595 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.825593 23595 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:42:02.825647 23595 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:42:02.825695 23595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:42:02.826633 23595 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.826750 23595 leader_election.cc:302] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf1bda79f514f7e82bbdc377a0abca8; no voters:
I20260806 06:42:02.826918 23595 leader_election.cc:288] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:42:02.826973 23599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:42:02.827104 23599 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [term 2 LEADER]: Becoming Leader. State: Replica: dbf1bda79f514f7e82bbdc377a0abca8, State: Running, Role: LEADER
I20260806 06:42:02.827191 23595 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:42:02.827214 23599 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } }
I20260806 06:42:02.827447 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } } }
I20260806 06:42:02.827445 23599 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf1bda79f514f7e82bbdc377a0abca8. Latest consensus state: current_term: 2 leader_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1bda79f514f7e82bbdc377a0abca8" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38307 } } }
I20260806 06:42:02.827661 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:02.827673 23599 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:02.828248 23608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:42:02.828708 23608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:42:02.828821 23608 catalog_manager.cc:1259] Loaded cluster ID: af938e45e9bf4d239f3d963cdf771277
I20260806 06:42:02.828860 23608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:42:02.830068 23608 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:42:02.830237 23608 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dbf1bda79f514f7e82bbdc377a0abca8: Loaded TSK: 0
I20260806 06:42:02.830432 23608 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:42:02.902639 23597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:02.902854 23597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:02.902880 23597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:02.904253 23597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:02.904300 23597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:02.904332 23597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:42:02.905913 23597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.65.keytab
--principal=oryx/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:34641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=36135
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:02.906149 23597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:02.906399 23597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:02.911689 23625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.911638 23623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:02.911648 23622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:02.912159 23597 server_base.cc:1034] running on GCE node
I20260806 06:42:02.912348 23597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:02.912632 23597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:02.913784 23597 hybrid_clock.cc:648] HybridClock initialized: now 1785998522913775 us; error 27 us; skew 500 ppm
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998522, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:02.916566 23597 init.cc:377] Logged in from keytab as oryx/127.14.51.65@KRBTEST.COM (short username oryx)
I20260806 06:42:02.917044 23597 webserver.cc:460] Webserver started at http://127.14.51.65:36135/ using document root <none> and password file <none>
I20260806 06:42:02.917273 23597 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:02.917352 23597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:02.918650 23597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:42:02.919282 23632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.919499 23597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:42:02.919598 23597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "f7b4404788a94e9d8088eade5b5257cf"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "defc79955950d7d189b1762e79b9229d"
server_key_iv: "c5869fa933fa13e9497ae39795d79aa4"
server_key_version: "encryptionkey@0"
I20260806 06:42:02.920015 23597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:02.969233 23597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:02.971237 23597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:02.971372 23597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:02.971551 23597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:02.971843 23597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:02.971873 23597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.971923 23597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:02.971949 23597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:02.977674 23597 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34641
I20260806 06:42:02.977720 23750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34641 every 8 connection(s)
I20260806 06:42:02.978046 23597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:42:02.982430 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23597
I20260806 06:42:02.982702 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:44741
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:38307
--webserver_port=45305
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:02.986254 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.978638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:40441)
0806 06:42:02.978792 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.978834 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.979389 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.979750 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.979753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.980300 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.980502 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.980506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.981507 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.981508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.982278 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.982280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.982948 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.982950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.983094 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.983595 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.983604 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.986122 (+ 2518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
W20260806 06:42:02.986481 23751 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:38307 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.51.126:38307: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
W20260806 06:42:02.986480 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:02.978703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:40441 (local address 127.14.51.126:38307)
0806 06:42:02.978871 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.978880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.978912 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:42:02.979550 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.979555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.979595 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.979702 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.980593 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.981416 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.982395 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.982846 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.982953 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.986286 (+ 3333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:40441: BlockingRecv error: recv got EOF from 127.14.51.65:40441 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:02.991520 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.986698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:43357)
0806 06:42:02.986805 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.986815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.986859 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.987005 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.987006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.987113 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.987199 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.987201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.988149 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.988150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.988618 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.988619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.989118 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.989119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.989166 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.989581 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.989585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.991479 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 06:42:02.991674 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:02.986762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43357 (local address 127.14.51.126:38307)
0806 06:42:02.986804 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.986807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.986849 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:42:02.986909 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.986910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.986923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.986999 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.987270 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.988069 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.988695 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.989032 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.989113 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.991597 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:43357: BlockingRecv error: recv got EOF from 127.14.51.65:43357 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:02 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:02.996250 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:02.991786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:54803)
0806 06:42:02.991842 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:02.991850 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:02.991880 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:02.992113 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:02.992114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:02.992213 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:02.992238 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.992239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.993113 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.993114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.993571 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:02.993572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.994045 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.994046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:02.994080 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.994477 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:02.994481 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:02.996187 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 06:42:02.996366 23751 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:42:02.996397 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:02.991849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:54803 (local address 127.14.51.126:38307)
0806 06:42:02.991990 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:02.991992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:02.992001 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:02.992023 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:02.992023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:02.992035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:02.992109 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:02.992308 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.993030 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.993647 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:02.993973 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:02.994060 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:02.996260 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:54803: BlockingRecv error: recv got EOF from 127.14.51.65:54803 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
W20260806 06:42:03.065514 23755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:03.065739 23755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:03.065773 23755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:03.066874 23755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:03.066936 23755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:03.066977 23755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:42:03.068377 23755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.66.keytab
--principal=oryx/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=45305
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:03.068632 23755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:03.068843 23755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:03.073489 23765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:03.073652 23761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:03.073734 23762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:03.073704 23755 server_base.cc:1034] running on GCE node
I20260806 06:42:03.073935 23755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:03.074179 23755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:03.075328 23755 hybrid_clock.cc:648] HybridClock initialized: now 1785998523075309 us; error 30 us; skew 500 ppm
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998523, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:03.077704 23755 init.cc:377] Logged in from keytab as oryx/127.14.51.66@KRBTEST.COM (short username oryx)
I20260806 06:42:03.078137 23755 webserver.cc:460] Webserver started at http://127.14.51.66:45305/ using document root <none> and password file <none>
I20260806 06:42:03.078361 23755 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:03.078438 23755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:03.079777 23755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:42:03.080307 23772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.080480 23755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:03.080576 23755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "007475be36aa420ebe874ea553a25131"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "ca52a536600837992fabac6d3021c65b"
server_key_iv: "d1e5cbb7d1b1d02b2de12e7e7de7f5d8"
server_key_version: "encryptionkey@0"
I20260806 06:42:03.080895 23755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:03.088986 23755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:03.091180 23755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:03.091331 23755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:03.091552 23755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:03.091838 23755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:03.091887 23755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.091920 23755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:03.091948 23755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.097666 23755 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:44741
I20260806 06:42:03.097718 23890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:44741 every 8 connection(s)
I20260806 06:42:03.098022 23755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:42:03.104847 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.098670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:42979 (local address 127.14.51.126:38307)
0806 06:42:03.098729 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.098732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.098904 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:42:03.099437 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.099438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.099456 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.099505 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.100347 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.100831 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.101787 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.102015 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.102096 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.104791 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:42979: BlockingRecv error: recv got EOF from 127.14.51.66:42979 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:42:03.104918 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.098643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:42979)
0806 06:42:03.098778 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.098828 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.099339 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.099579 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.099582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.100089 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.100255 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.100259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.100938 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.100939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.101705 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.101706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.102106 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.102107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.102223 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.102616 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.102623 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.104724 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
W20260806 06:42:03.105110 23891 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:38307 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.51.126:38307: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
I20260806 06:42:03.108093 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23755
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:03.108392 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:43347
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:38307
--webserver_port=36071
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:03.109130 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.105314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:57685)
0806 06:42:03.105355 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.105366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.105424 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.105538 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.105539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.105659 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.105696 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.105697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.106276 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.106277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.106764 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.106765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.107114 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.107115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.107194 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.107493 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.107497 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.109091 (+ 1594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:42:03.109256 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.105373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:57685 (local address 127.14.51.126:38307)
0806 06:42:03.105436 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.105437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.105446 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:03.105469 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.105470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.105479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.105534 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.105772 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.106206 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.106842 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.107036 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.107113 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.109178 (+ 2065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:57685: BlockingRecv error: recv got EOF from 127.14.51.66:57685 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:03.113516 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.109399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:51339)
0806 06:42:03.109479 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.109487 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.109517 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.109636 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.109637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.109790 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.109823 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.109825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.110353 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.110354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.110995 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.110997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.111467 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.111467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.111503 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.111866 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.111871 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.113479 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 06:42:03.113653 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.109449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:51339 (local address 127.14.51.126:38307)
0806 06:42:03.109497 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.109499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.109511 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:42:03.109562 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.109563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.109572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.109632 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.109891 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.110285 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.111073 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.111399 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.111467 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.113579 (+ 2112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:51339: BlockingRecv error: recv got EOF from 127.14.51.66:51339 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 06:42:03.113708 23891 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:42:03.189582 23894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:03.189790 23894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:03.189838 23894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:03.191298 23894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:03.191345 23894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:03.191390 23894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:42:03.192768 23894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/krb5kdc/oryx_127.14.51.67.keytab
--principal=oryx/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:43347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=36071
--tserver_master_addrs=127.14.51.126:38307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:03.192989 23894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:03.193208 23894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:03.197922 23899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:03.198066 23894 server_base.cc:1034] running on GCE node
W20260806 06:42:03.197937 23900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:03.197942 23902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:03.198343 23894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:03.198527 23894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:03.199677 23894 hybrid_clock.cc:648] HybridClock initialized: now 1785998523199660 us; error 29 us; skew 500 ppm
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998523, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:03.201959 23894 init.cc:377] Logged in from keytab as oryx/127.14.51.67@KRBTEST.COM (short username oryx)
I20260806 06:42:03.202345 23894 webserver.cc:460] Webserver started at http://127.14.51.67:36071/ using document root <none> and password file <none>
I20260806 06:42:03.202550 23894 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:03.202628 23894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:03.204006 23894 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:42:03.204535 23909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.204690 23894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:03.204782 23894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "6734e9ce946e47f4855027f3d8937b2c"
format_stamp: "Formatted at 2026-08-06 06:42:02 on dist-test-slave-1xrh"
server_key: "a8f7caddaccbbbfa9e40fee39f391782"
server_key_iv: "c4c81f766819440a695dc86644243047"
server_key_version: "encryptionkey@0"
I20260806 06:42:03.205091 23894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:03.223500 23894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:03.225371 23894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:03.225519 23894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:03.225726 23894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:03.226016 23894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:03.226065 23894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.226106 23894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:03.226135 23894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:03.232077 23894 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:43347
I20260806 06:42:03.232143 24028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:43347 every 8 connection(s)
I20260806 06:42:03.232487 23894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:42:03.233584 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 23894
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:42:03.240446 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.233167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:60151 (local address 127.14.51.126:38307)
0806 06:42:03.233226 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.233228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.233418 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:42:03.234015 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.234016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.234030 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.234092 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.235199 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.235700 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.236777 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.237094 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.237145 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.240365 (+ 3220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:60151: BlockingRecv error: recv got EOF from 127.14.51.67:60151 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:42:03.240537 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.233133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:60151)
0806 06:42:03.233292 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.233335 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.233906 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.234216 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.234220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.234850 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.235089 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.235094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.235786 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.235787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.236705 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.236708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.237170 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.237171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.237305 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.237782 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.237789 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.240301 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
W20260806 06:42:03.240912 24029 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:38307 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.51.126:38307: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
I20260806 06:42:03.241587 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.234101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.14.51.126:38307)
0806 06:42:03.234163 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.234165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.234188 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:42:03.234261 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.234262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.234274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.234351 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.234958 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.235597 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.236380 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.236669 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.236737 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.238045 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:03.238050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:03.238050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:03.238061 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:03.240006 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:03.240445 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:03.240449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:03.240449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:03.240609 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:03.240823 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:03.240824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:03.240824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:03.241070 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:03.241209 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:03.241364 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:42:03.241468 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:03.245224 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.241138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:56681)
0806 06:42:03.241179 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.241188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.241222 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.241362 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.241362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.241505 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.241556 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.241557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.242288 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.242289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.242815 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.242816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.243182 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.243183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.243247 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.243614 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.243618 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.245153 (+ 1535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:42:03.245276 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.241176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:56681 (local address 127.14.51.126:38307)
0806 06:42:03.241209 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.241210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.241219 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:03.241284 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.241285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.241293 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.241342 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.241618 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.242210 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.242889 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.243083 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.243181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.245217 (+ 2036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:56681: BlockingRecv error: recv got EOF from 127.14.51.67:56681 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:03.249104 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.245500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:42819)
0806 06:42:03.245545 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.245556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.245597 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.245695 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.245695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.245790 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.245812 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.245813 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.246321 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.246321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.246732 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.246733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.247228 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.247229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.247262 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.247639 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.247644 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:03.249065 (+ 1421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:42:03.249208 24029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:42:03.249279 23754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.245563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:42819 (local address 127.14.51.126:38307)
0806 06:42:03.245610 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.245611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.245618 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:42:03.245636 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.245637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.245643 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.245691 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.245867 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.246267 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.246926 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.247094 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.247227 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.249203 (+ 1976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:42819: BlockingRecv error: recv got EOF from 127.14.51.67:42819 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:03 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:04.001093 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:03.996699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:34679)
0806 06:42:03.996856 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:03.996873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:03.996942 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:03.997123 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:03.997124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:03.997281 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:03.997330 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.997332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.997949 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.997949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.998474 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:03.998475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.998791 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.998792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:03.998830 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:03.999354 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:03.999359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:04.001026 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
W20260806 06:42:04.001191 24035 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:03.996860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:34679 (local address 127.14.51.126:38307)
0806 06:42:03.996968 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:03.996969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:03.996978 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:03.997007 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:03.997008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:03.997017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:03.997095 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:03.997390 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.997884 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.998539 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:03.998730 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:03.998785 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:04.001120 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:34679: BlockingRecv error: recv got EOF from 127.14.51.65:34679 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
Aug 06 06:42:04 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:04 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:04.118638 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:04.114044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:39747)
0806 06:42:04.114202 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:04.114214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:04.114291 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:04.114422 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:04.114423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:04.114600 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:04.114645 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:04.114647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.115374 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.115375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:04.115895 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:04.115896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.116251 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.116252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:04.116329 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:04.116778 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:04.116782 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:04.118532 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
W20260806 06:42:04.118685 24035 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:04.114125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:39747 (local address 127.14.51.126:38307)
0806 06:42:04.114195 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:04.114197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:04.114272 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:42:04.114353 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:04.114353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:04.114367 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:04.114418 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:04.114722 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.115299 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.115983 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.116185 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.116255 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:04.118617 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:39747: BlockingRecv error: recv got EOF from 127.14.51.66:39747 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 06:42:04 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:04 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:04.254348 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:04.249603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:51057)
0806 06:42:04.249789 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:04.249814 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:04.249897 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:04.250135 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:04.250137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:04.250358 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:04.250419 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:04.250422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.251075 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.251076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:04.251689 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:04.251690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.252003 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.252004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:04.252042 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:04.252527 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:04.252538 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:04.254284 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
W20260806 06:42:04.254454 24035 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:04.249700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:51057 (local address 127.14.51.126:38307)
0806 06:42:04.249776 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:04.249778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:04.249878 (+ 100us) server_negotiation.cc:407] Connection header received
0806 06:42:04.249973 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:04.249974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:04.250003 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:04.250083 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:04.250504 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.251009 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.251757 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:04.251942 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:04.251996 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:04.254365 (+ 2369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:51057: BlockingRecv error: recv got EOF from 127.14.51.67:51057 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:05.005874 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:05.001522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:54889)
0806 06:42:05.001633 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:05.001645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:05.001705 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:05.001830 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:05.001831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:05.001986 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:05.002037 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.002039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.002659 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.002660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.003209 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.003210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.003529 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.003530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.003571 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.004026 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:05.004030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:05.005816 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":56,"threads_started":1}
W20260806 06:42:05.006009 24039 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:05.001583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:54889 (local address 127.14.51.126:38307)
0806 06:42:05.001721 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:05.001722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:05.001730 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:42:05.001752 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:05.001752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:05.001762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:05.001818 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:05.002107 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.002597 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.003275 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.003470 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.003521 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.005920 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:54889: BlockingRecv error: recv got EOF from 127.14.51.65:54889 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:05.123457 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:05.119086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:48283)
0806 06:42:05.119222 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:05.119232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:05.119286 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:05.119406 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:05.119407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:05.119562 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:05.119622 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.119624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.120253 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.120254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.120748 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.120749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.121111 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.121111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.121149 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.121604 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:05.121608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:05.123401 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
W20260806 06:42:05.123589 24039 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:05.119139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:48283 (local address 127.14.51.126:38307)
0806 06:42:05.119210 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:05.119212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:05.119272 (+ 60us) server_negotiation.cc:407] Connection header received
0806 06:42:05.119341 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:05.119342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:05.119352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:05.119392 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:05.119695 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.120191 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.120826 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.121041 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.121104 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.123509 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:48283: BlockingRecv error: recv got EOF from 127.14.51.66:48283 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:05 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:05.259440 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:05.254851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:54431)
0806 06:42:05.254997 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:05.255007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:05.255075 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:05.255263 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:05.255264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:05.255428 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:05.255477 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.255479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.256088 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.256089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.256825 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:05.256828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.257166 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.257168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:05.257235 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.257640 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:05.257644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:05.259384 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
W20260806 06:42:05.259569 24039 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:05.255075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:54431 (local address 127.14.51.126:38307)
0806 06:42:05.255150 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:05.255151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:05.255160 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:05.255180 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:05.255181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:05.255193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:05.255246 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:05.255533 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.256002 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.256890 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:05.257089 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:05.257163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:05.259467 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:54431: BlockingRecv error: recv got EOF from 127.14.51.67:54431 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:06.010777 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:06.006374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:46175)
0806 06:42:06.006502 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:06.006512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:06.006569 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:06.006683 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:06.006684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:06.006848 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:06.006898 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.006900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.007569 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.007570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.008142 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.008144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.008473 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.008474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.008513 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.008999 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:06.009003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:06.010718 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
W20260806 06:42:06.010893 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:06.006426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:46175 (local address 127.14.51.126:38307)
0806 06:42:06.006550 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:06.006552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:06.006561 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:06.006612 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:06.006612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:06.006623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:06.006679 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:06.006953 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.007481 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.008211 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.008412 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.008465 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.010811 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:46175: BlockingRecv error: recv got EOF from 127.14.51.65:46175 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:06.128574 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:06.123946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:41961)
0806 06:42:06.124106 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:06.124116 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:06.124180 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:06.124367 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:06.124369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:06.124582 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:06.124632 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.124634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.125276 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.125277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.125779 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.125780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.126114 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.126115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.126159 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.126635 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:06.126639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:06.128479 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
W20260806 06:42:06.128615 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:06.124000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:41961 (local address 127.14.51.126:38307)
0806 06:42:06.124060 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:06.124062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:06.124176 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:42:06.124256 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:06.124257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:06.124275 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:06.124343 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:06.124698 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.125217 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.125861 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.126052 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.126101 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.128555 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:41961: BlockingRecv error: recv got EOF from 127.14.51.66:41961 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:06 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:06.264573 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:06.260056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:44359)
0806 06:42:06.260192 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:06.260208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:06.260263 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:06.260390 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:06.260391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:06.260567 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:06.260616 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.260618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.261207 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.261207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.261713 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:06.261714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.261998 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.261999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:06.262034 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.262509 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:06.262514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:06.264480 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
W20260806 06:42:06.264642 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:06.260148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:44359 (local address 127.14.51.126:38307)
0806 06:42:06.260208 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:06.260210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:06.260247 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:42:06.260316 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:06.260316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:06.260328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:06.260376 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:06.260673 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.261146 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.261774 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:06.261943 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:06.261989 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:06.264577 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:44359: BlockingRecv error: recv got EOF from 127.14.51.67:44359 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:07.015861 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:07.011498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:56735)
0806 06:42:07.011649 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:07.011659 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:07.011700 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:07.011809 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:07.011809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:07.011972 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:07.012022 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.012024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.012628 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.012629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.013170 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.013172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.013480 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.013481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.013520 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.013957 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:07.013961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:07.015801 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
W20260806 06:42:07.015990 24047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:07.011561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:56735 (local address 127.14.51.126:38307)
0806 06:42:07.011676 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:07.011678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:07.011705 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:42:07.011744 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:07.011745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:07.011756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:07.011799 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:07.012084 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.012565 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.013236 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.013422 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.013470 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.015898 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:56735: BlockingRecv error: recv got EOF from 127.14.51.65:56735 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:07.137935 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:07.129018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:56007)
0806 06:42:07.129206 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:07.129223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:07.129303 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:07.129466 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:07.129467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:07.129647 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:07.129694 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.129695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.130353 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.130354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.130871 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.130872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.131285 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.131286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.131351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.135801 (+ 4450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:07.135806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:07.137834 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
W20260806 06:42:07.137987 24047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:07.129094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:56007 (local address 127.14.51.126:38307)
0806 06:42:07.129187 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:07.129190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:07.129282 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:42:07.129383 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:07.129384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:07.129400 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:07.129460 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:07.129777 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.130275 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.130953 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.131206 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.131284 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.137922 (+ 6638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:56007: BlockingRecv error: recv got EOF from 127.14.51.66:56007 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:07 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:07.269621 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:07.265082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:57759)
0806 06:42:07.265259 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:07.265268 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:07.265351 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:07.265495 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:07.265497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:07.265680 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:07.265726 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.265727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.266361 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.266362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.266884 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:07.266885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.267318 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.267319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:07.267394 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.267816 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:07.267821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:07.269565 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
W20260806 06:42:07.269742 24047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:07.265144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:57759 (local address 127.14.51.126:38307)
0806 06:42:07.265215 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:07.265217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:07.265340 (+ 123us) server_negotiation.cc:407] Connection header received
0806 06:42:07.265404 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:07.265404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:07.265420 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:07.265488 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:07.265799 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.266276 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.266957 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:07.267230 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:07.267317 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:07.269647 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:57759: BlockingRecv error: recv got EOF from 127.14.51.67:57759 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:08.020642 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:08.016321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:55713)
0806 06:42:08.016465 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:08.016475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:08.016521 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:08.016679 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:08.016680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:08.016843 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:08.016894 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.016896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.017519 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.017520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.018064 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.018065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.018378 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.018379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.018426 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.018872 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:08.018876 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:08.020585 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260806 06:42:08.020763 24051 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:08.016416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:55713 (local address 127.14.51.126:38307)
0806 06:42:08.016554 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:08.016556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:08.016564 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:42:08.016586 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:08.016586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:08.016597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:08.016662 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:08.016959 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.017441 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.018134 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.018317 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.018368 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.020679 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:55713: BlockingRecv error: recv got EOF from 127.14.51.65:55713 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:08.143257 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:08.138422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:46255)
0806 06:42:08.138658 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:08.138671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:08.138739 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:08.138874 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:08.138874 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:08.139040 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:08.139089 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.139091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.139793 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.139794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.140318 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.140320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.140641 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.140642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.140679 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.141143 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:08.141147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:08.143189 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":64,"threads_started":1}
W20260806 06:42:08.143289 24051 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:08.138675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:46255 (local address 127.14.51.126:38307)
0806 06:42:08.138729 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:08.138731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:08.138742 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:42:08.138790 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:08.138791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:08.138801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:08.138861 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:08.139212 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.139718 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.140384 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.140568 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.140631 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.143244 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:46255: BlockingRecv error: recv got EOF from 127.14.51.66:46255 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:08 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:08.275182 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:08.270100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:60903)
0806 06:42:08.270305 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:08.270319 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:08.270415 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:08.270594 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:08.270595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:08.270780 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:08.270829 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.270831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.271622 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.271623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.272153 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:08.272155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.272517 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.272518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:08.272585 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.273082 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:08.273087 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:08.275056 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
W20260806 06:42:08.275215 24051 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:08.270168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:60903 (local address 127.14.51.126:38307)
0806 06:42:08.270255 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:08.270257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:08.270400 (+ 143us) server_negotiation.cc:407] Connection header received
0806 06:42:08.270482 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:08.270483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:08.270497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:08.270578 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:08.270912 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.271483 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.272243 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:08.272433 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:08.272506 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:08.275154 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:60903: BlockingRecv error: recv got EOF from 127.14.51.67:60903 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:09.027974 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:09.021134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:48387)
0806 06:42:09.021256 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:09.021267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:09.021348 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:09.021574 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:09.021575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:09.021863 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:09.021914 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.021916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.022817 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.022819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.023905 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.023909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.024491 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.024492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.024551 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.025122 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:09.025129 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:09.027853 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260806 06:42:09.028122 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:09.021260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:48387 (local address 127.14.51.126:38307)
0806 06:42:09.021430 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:09.021432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:09.021442 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:42:09.021468 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:09.021468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:09.021483 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:09.021567 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:09.022014 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.022710 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.024015 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.024383 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.024475 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.027964 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:48387: BlockingRecv error: recv got EOF from 127.14.51.65:48387 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:09.148489 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:09.143840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:54945)
0806 06:42:09.143998 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:09.144010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:09.144077 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:09.144204 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:09.144204 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:09.144394 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:09.144441 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.144443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.145129 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.145130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.145642 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.145644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.146003 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.146004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.146061 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.146496 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:09.146500 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:09.148393 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
W20260806 06:42:09.148535 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:09.143877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:54945 (local address 127.14.51.126:38307)
0806 06:42:09.143938 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:09.143941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:09.144066 (+ 125us) server_negotiation.cc:407] Connection header received
0806 06:42:09.144135 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:09.144136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:09.144150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:09.144200 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:09.144523 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.145034 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.145727 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.145932 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.146006 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.148470 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:54945: BlockingRecv error: recv got EOF from 127.14.51.66:54945 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:09 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:09.280082 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:09.275637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:48483)
0806 06:42:09.275775 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:09.275785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:09.275840 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:09.275949 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:09.275950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:09.276117 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:09.276165 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.276167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.276812 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.276813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.277332 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:09.277333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.277658 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.277659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:09.277697 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.278193 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:09.278197 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:09.280019 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260806 06:42:09.280211 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:09.275719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:48483 (local address 127.14.51.126:38307)
0806 06:42:09.275784 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:09.275787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:09.275828 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:42:09.275893 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:09.275894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:09.275905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:09.275945 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:09.276226 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.276741 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.277403 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:09.277599 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:09.277649 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:09.280103 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:48483: BlockingRecv error: recv got EOF from 127.14.51.67:48483 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:10.033517 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:10.028654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:50637)
0806 06:42:10.028818 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:10.028829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:10.028946 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:10.029090 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:10.029091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:10.029270 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:10.029322 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.029324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.029974 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.029975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.030517 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.030518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.030966 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.030967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.031028 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.031549 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:10.031554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:10.033417 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
W20260806 06:42:10.033579 24060 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:10.028731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:50637 (local address 127.14.51.126:38307)
0806 06:42:10.028879 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:10.028881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:10.028937 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:42:10.028991 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:10.028992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:10.029004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:10.029064 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:10.029395 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.029895 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.030609 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.030888 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.030965 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.033504 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:50637: BlockingRecv error: recv got EOF from 127.14.51.65:50637 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:10.153705 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:10.149053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:38501)
0806 06:42:10.149219 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:10.149232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:10.149323 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:10.149518 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:10.149519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:10.149721 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:10.149767 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.149768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.150473 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.150474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.151017 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.151019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.151415 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.151416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.151475 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.151895 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:10.151899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:10.153615 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
W20260806 06:42:10.153754 24060 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:10.149132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:38501 (local address 127.14.51.126:38307)
0806 06:42:10.149204 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:10.149206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:10.149304 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:42:10.149428 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:10.149429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:10.149442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:10.149510 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:10.149890 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.150385 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.151108 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.151334 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.151415 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.153692 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:38501: BlockingRecv error: recv got EOF from 127.14.51.66:38501 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:10 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:10.285457 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:10.280689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:33241)
0806 06:42:10.280860 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:10.280871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:10.280935 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:10.281051 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:10.281051 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:10.281207 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:10.281254 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.281256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.281938 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.281939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.282484 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:10.282486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.282801 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.282801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:10.282840 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.283400 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:10.283407 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:10.285399 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
W20260806 06:42:10.285607 24060 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:10.280742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:33241 (local address 127.14.51.126:38307)
0806 06:42:10.280848 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:10.280851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:10.280946 (+ 95us) server_negotiation.cc:407] Connection header received
0806 06:42:10.280972 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:10.280973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:10.280984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:10.281038 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:10.281348 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.281852 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.282559 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:10.282737 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:10.282788 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:10.285519 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:33241: BlockingRecv error: recv got EOF from 127.14.51.67:33241 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:11.038511 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:11.034059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:58575)
0806 06:42:11.034225 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:11.034238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:11.034309 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:11.034455 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:11.034456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:11.034625 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:11.034674 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.034675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.035373 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.035373 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.035897 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.035906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.036236 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.036237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.036275 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.036690 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:11.036694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:11.038444 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
W20260806 06:42:11.038651 24064 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:11.034189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:58575 (local address 127.14.51.126:38307)
0806 06:42:11.034325 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:11.034327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:11.034350 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:42:11.034370 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:11.034371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:11.034381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:11.034433 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:11.034739 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.035296 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.035977 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.036173 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.036227 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.038569 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:58575: BlockingRecv error: recv got EOF from 127.14.51.65:58575 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:11.159072 24065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:11.154289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:41263)
0806 06:42:11.154436 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:11.154448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:11.154519 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:11.154651 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:11.154652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:11.154857 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:11.154906 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.154907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.155677 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.155678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.156297 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.156299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.156649 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.156650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.156700 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.157263 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:11.157268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:11.159003 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
W20260806 06:42:11.159188 24064 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:11.154339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:41263 (local address 127.14.51.126:38307)
0806 06:42:11.154420 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:11.154421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:11.154519 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:42:11.154579 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:11.154579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:11.154590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:11.154629 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:11.154965 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.155610 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.156375 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.156560 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.156610 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.159090 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:41263: BlockingRecv error: recv got EOF from 127.14.51.66:41263 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:11 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:11.290526 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:11.286008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:57503)
0806 06:42:11.286149 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:11.286162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:11.286226 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:11.286350 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:11.286350 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:11.286529 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:11.286587 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.286589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.287267 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.287268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.287812 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:11.287814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.288157 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.288157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:11.288202 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.288624 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:11.288628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:11.290455 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
W20260806 06:42:11.290617 24064 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:11.286048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:57503 (local address 127.14.51.126:38307)
0806 06:42:11.286107 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:11.286109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:11.286227 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:42:11.286281 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:11.286281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:11.286293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:11.286333 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:11.286648 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.287185 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.287900 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:11.288094 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:11.288143 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:11.290552 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:57503: BlockingRecv error: recv got EOF from 127.14.51.67:57503 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:12.043545 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:12.039017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:43469)
0806 06:42:12.039237 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:12.039247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:12.039303 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:12.039441 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:12.039442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:12.039630 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:12.039685 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.039687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.040352 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.040353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.040889 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.040890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.041229 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.041230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.041273 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.041705 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:12.041709 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:12.043488 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":133,"threads_started":1}
W20260806 06:42:12.043660 24068 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:12.039093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43469 (local address 127.14.51.126:38307)
0806 06:42:12.039260 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:12.039262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:12.039291 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:42:12.039360 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:12.039361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:12.039371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:12.039425 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:12.039749 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.040274 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.040960 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.041148 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.041203 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.043583 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:43469: BlockingRecv error: recv got EOF from 127.14.51.65:43469 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:12.168608 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:12.159522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:37347)
0806 06:42:12.159693 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:12.159710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:12.159773 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:12.159883 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:12.159883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:12.160058 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:12.160107 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.160109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.160736 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.160737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.161259 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.161260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.161582 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.161583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.161621 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.166697 (+ 5076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:12.166702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:12.168548 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":122,"threads_started":1}
W20260806 06:42:12.168742 24068 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:12.159614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:37347 (local address 127.14.51.126:38307)
0806 06:42:12.159667 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:12.159668 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:12.159770 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:42:12.159816 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:12.159817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:12.159828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:12.159872 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:12.160161 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.160668 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.161341 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.161519 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.161568 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.168659 (+ 7091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:37347: BlockingRecv error: recv got EOF from 127.14.51.66:37347 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:12 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:12.296144 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:12.290999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:46171)
0806 06:42:12.291187 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:12.291199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:12.291278 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:12.291427 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:12.291429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:12.291605 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:12.291654 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.291656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.292506 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.292507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.293318 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:12.293321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.293679 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.293680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:12.293738 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.294251 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:12.294256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:12.296087 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
W20260806 06:42:12.296276 24068 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:12.291079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46171 (local address 127.14.51.126:38307)
0806 06:42:12.291170 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:12.291173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:12.291271 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:42:12.291350 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:12.291351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:12.291365 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:12.291414 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:12.291714 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.292427 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.293393 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:12.293597 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:12.293677 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:12.296195 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:46171: BlockingRecv error: recv got EOF from 127.14.51.67:46171 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:13.048952 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:13.044048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:51047)
0806 06:42:13.044200 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:13.044212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:13.044260 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:13.044459 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:13.044460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:13.044630 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:13.044680 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.044681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.045618 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.045619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.046143 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.046144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.046499 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.046500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.046539 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.047021 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:13.047026 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:13.048885 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
W20260806 06:42:13.049126 24072 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:13.044182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:51047 (local address 127.14.51.126:38307)
0806 06:42:13.044322 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:13.044324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:13.044332 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:42:13.044362 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:13.044363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:13.044374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:13.044447 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:13.044750 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.045543 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.046213 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.046430 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.046494 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.049026 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:51047: BlockingRecv error: recv got EOF from 127.14.51.65:51047 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:13.173972 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:13.169073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:49437)
0806 06:42:13.169253 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:13.169265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:13.169349 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:13.169516 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:13.169518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:13.169703 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:13.169750 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.169752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.170434 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.170435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.170966 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.170967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.171390 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.171391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.171459 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.171966 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:13.171972 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:13.173911 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260806 06:42:13.174103 24072 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:13.169163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:49437 (local address 127.14.51.126:38307)
0806 06:42:13.169240 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:13.169242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:13.169334 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:42:13.169422 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:13.169423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:13.169441 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:13.169510 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:13.169829 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.170332 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.171050 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.171308 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.171381 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.174022 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:49437: BlockingRecv error: recv got EOF from 127.14.51.66:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:13 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:13.301460 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:13.296685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:39155)
0806 06:42:13.296838 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:13.296875 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:13.296978 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:13.297142 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:13.297143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:13.297328 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:13.297374 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.297376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.298046 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.298047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.298585 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:13.298586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.298954 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.298955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:13.299017 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.299531 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:13.299535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:13.301377 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
W20260806 06:42:13.301498 24072 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:13.296807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:39155 (local address 127.14.51.126:38307)
0806 06:42:13.296901 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:13.296903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:13.296971 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:42:13.297043 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:13.297044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:13.297060 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:13.297136 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:13.297453 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.297963 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.298671 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:13.298866 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:13.298954 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:13.301451 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:39155: BlockingRecv error: recv got EOF from 127.14.51.67:39155 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:14.054606 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:14.049480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:57855)
0806 06:42:14.049673 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:14.049688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:14.049762 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:14.049926 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:14.049927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:14.050164 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:14.050234 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.050236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.051033 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.051034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.051763 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.051766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.052133 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.052134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.052192 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.052718 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:14.052723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:14.054536 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
W20260806 06:42:14.054675 24076 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:14.049565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:57855 (local address 127.14.51.126:38307)
0806 06:42:14.049734 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:14.049736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:14.049745 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:14.049825 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:14.049826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:14.049837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:14.049888 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:14.050321 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.050952 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.051859 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.052061 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.052135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.054623 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:57855: BlockingRecv error: recv got EOF from 127.14.51.65:57855 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:14.179116 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:14.174532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:60891)
0806 06:42:14.174711 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:14.174722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:14.174778 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:14.174900 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:14.174900 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:14.175113 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:14.175216 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.175219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.175860 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.175861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.176389 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.176391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.176727 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.176728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.176771 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.177218 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:14.177223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:14.179003 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
W20260806 06:42:14.179184 24076 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:14.174590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:60891 (local address 127.14.51.126:38307)
0806 06:42:14.174665 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:14.174668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:14.174779 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:42:14.174839 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:14.174840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:14.174851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:14.174888 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:14.175281 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.175790 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.176459 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.176661 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.176713 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.179088 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:60891: BlockingRecv error: recv got EOF from 127.14.51.66:60891 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:14 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:14.306484 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:14.301975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:38617)
0806 06:42:14.302159 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:14.302169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:14.302229 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:14.302421 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:14.302422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:14.302603 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:14.302651 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.302653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.303320 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.303321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.303858 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:14.303859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.304179 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.304180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:14.304219 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.304649 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:14.304653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:14.306420 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
W20260806 06:42:14.306609 24076 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:14.302186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:38617 (local address 127.14.51.126:38307)
0806 06:42:14.302277 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:14.302279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:14.302288 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:14.302330 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:14.302331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:14.302347 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:14.302404 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:14.302751 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.303254 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.303936 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:14.304116 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:14.304168 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:14.306500 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:38617: BlockingRecv error: recv got EOF from 127.14.51.67:38617 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:15.059458 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:15.055084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:58699)
0806 06:42:15.055236 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:15.055246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:15.055304 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:15.055442 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:15.055443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:15.055610 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:15.055657 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.055658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.056262 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.056263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.056796 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.056797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.057121 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.057122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.057161 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.057600 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:15.057604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:15.059400 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260806 06:42:15.059648 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:15.055156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:58699 (local address 127.14.51.126:38307)
0806 06:42:15.055303 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:15.055307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:15.055323 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:42:15.055346 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:15.055346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:15.055356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:15.055420 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:15.055722 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.056201 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.056864 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.057045 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.057105 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.059528 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:58699: BlockingRecv error: recv got EOF from 127.14.51.65:58699 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:15.184587 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:15.179726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:53509)
0806 06:42:15.179880 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:15.179892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:15.179959 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:15.180084 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:15.180085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:15.180253 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:15.180298 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.180299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.180902 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.180903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.181660 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.181664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.182060 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.182061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.182104 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.182596 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:15.182602 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:15.184522 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
W20260806 06:42:15.184748 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:15.179783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:53509 (local address 127.14.51.126:38307)
0806 06:42:15.179854 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:15.179856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:15.179950 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:42:15.180016 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:15.180017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:15.180029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:15.180078 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:15.180353 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.180834 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.181743 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.181998 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.182050 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.184663 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:53509: BlockingRecv error: recv got EOF from 127.14.51.66:53509 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:15 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:15.312425 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:15.306948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:47175)
0806 06:42:15.307158 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:15.307171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:15.307235 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:15.307381 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:15.307381 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:15.307597 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:15.307643 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.307645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.308332 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.308333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.308933 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:15.308935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.309305 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.309307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:15.309365 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.309953 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:15.309959 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:15.312331 (+ 2372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
W20260806 06:42:15.312481 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:15.307107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:47175 (local address 127.14.51.126:38307)
0806 06:42:15.307203 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:15.307205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:15.307217 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:42:15.307312 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:15.307313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:15.307327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:15.307377 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:15.307723 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.308240 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.309019 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:15.309230 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:15.309308 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:15.312413 (+ 3105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:47175: BlockingRecv error: recv got EOF from 127.14.51.67:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:16.064777 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:16.059965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:48063)
0806 06:42:16.060103 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:16.060114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:16.060163 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:16.060303 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:16.060304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:16.060503 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:16.060552 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.060553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.061179 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.061179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.061825 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.061829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.062259 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.062260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.062345 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.062803 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:16.062807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:16.064683 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260806 06:42:16.064886 24084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:16.060051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:48063 (local address 127.14.51.126:38307)
0806 06:42:16.060173 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:16.060175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:16.060183 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:42:16.060212 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:16.060213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:16.060224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:16.060291 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:16.060620 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.061115 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.061891 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.062161 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.062260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.064809 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:48063: BlockingRecv error: recv got EOF from 127.14.51.65:48063 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:16.189571 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:16.185023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:39575)
0806 06:42:16.185202 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:16.185214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:16.185300 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:16.185422 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:16.185424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:16.185587 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:16.185644 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.185645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.186289 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.186290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.186853 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.186855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.187221 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.187221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.187265 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.187701 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:16.187705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:16.189504 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
W20260806 06:42:16.189638 24084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:16.185162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:39575 (local address 127.14.51.126:38307)
0806 06:42:16.185208 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:16.185210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:16.185304 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:42:16.185353 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:16.185354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:16.185364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:16.185404 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:16.185702 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.186210 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.186937 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.187148 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.187207 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.189593 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:39575: BlockingRecv error: recv got EOF from 127.14.51.66:39575 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:16 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:16.320025 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:16.313158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:37529)
0806 06:42:16.313307 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:16.313317 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:16.313467 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:16.313643 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:16.313643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:16.313931 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:16.314004 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.314007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.315168 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.315170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.316236 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:16.316239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.316828 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.316830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:16.316893 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.317475 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:16.317480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:16.319928 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
W20260806 06:42:16.320129 24084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:16.313231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:37529 (local address 127.14.51.126:38307)
0806 06:42:16.313338 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:16.313341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:16.313468 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:42:16.313534 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:16.313535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:16.313554 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:16.313637 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:16.314101 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.315054 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.316334 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:16.316735 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:16.316815 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:16.320050 (+ 3235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:37529: BlockingRecv error: recv got EOF from 127.14.51.67:37529 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:17.070681 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:17.065353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:36879)
0806 06:42:17.065489 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:17.065499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:17.065608 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:17.065795 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:17.065795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:17.066005 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:17.066054 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.066056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.066684 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.066684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.067258 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.067260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.067654 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.067655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.067721 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.068312 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:17.068317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:17.070561 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
W20260806 06:42:17.070735 24089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:17.065415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:36879 (local address 127.14.51.126:38307)
0806 06:42:17.065529 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:17.065531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:17.065587 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:42:17.065675 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:17.065676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:17.065689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:17.065778 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:17.066120 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.066611 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.067350 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.067566 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.067652 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.070657 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:36879: BlockingRecv error: recv got EOF from 127.14.51.65:36879 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":47,"threads_started":1}
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:17.199452 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:17.190034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:41167)
0806 06:42:17.190212 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:17.190229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:17.190308 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:17.190480 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:17.190481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:17.190707 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:17.190753 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.190755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.191467 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.191468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.191979 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.191980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.192346 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.192347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.192404 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.197508 (+ 5104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:17.197512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:17.199356 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
W20260806 06:42:17.199491 24089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:17.190112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:41167 (local address 127.14.51.126:38307)
0806 06:42:17.190196 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:17.190198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:17.190287 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:42:17.190402 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:17.190403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:17.190417 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:17.190465 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:17.190827 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.191388 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.192079 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.192274 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.192348 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.199431 (+ 7083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:41167: BlockingRecv error: recv got EOF from 127.14.51.66:41167 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:17 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.67@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:17.325944 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:17.320559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.67:46107)
0806 06:42:17.320727 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:17.320743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:17.320835 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:17.321017 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:17.321018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:17.321229 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:17.321278 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.321283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.322057 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.322058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.322894 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:17.322897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.323390 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.323391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:17.323456 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.323968 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:17.323973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:17.325884 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260806 06:42:17.326048 24089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:17.320663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46107 (local address 127.14.51.126:38307)
0806 06:42:17.320746 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:17.320748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:17.320803 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:42:17.320905 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:17.320906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:17.320921 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:17.321006 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:17.321384 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.321960 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.322988 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:17.323279 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:17.323388 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:17.325973 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.67:46107: BlockingRecv error: recv got EOF from 127.14.51.67:46107 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 06:42:18 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:18 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.65@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:18.076505 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.071262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.65:40411)
0806 06:42:18.071390 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:18.071401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:18.071495 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:18.071653 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:18.071654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:18.071890 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:18.071951 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.071957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.072927 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.072929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.073894 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.073898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.074273 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.074274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.074324 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.074743 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:18.074747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:18.076447 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260806 06:42:18.076643 24094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:18.071320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:40411 (local address 127.14.51.126:38307)
0806 06:42:18.071479 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.071481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.071490 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:18.071557 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.071558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.071572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.071643 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.072059 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.072833 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.073994 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.074204 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.074258 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.076556 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.65:40411: BlockingRecv error: recv got EOF from 127.14.51.65:40411 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
Aug 06 06:42:18 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:42:18 dist-test-slave-1xrh krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.51.66@KRBTEST.COM for oryx/127.14.51.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:42:18.204907 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.199947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38307 (local address 127.14.51.66:54199)
0806 06:42:18.200078 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:18.200090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:18.200155 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:18.200282 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:18.200282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:18.200489 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:18.200552 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.200555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.201670 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.201672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.202210 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.202212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.202580 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.202580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.202621 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.203076 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:18.203080 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:18.204840 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.51.126:38307: Server oryx/127.14.51.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
W20260806 06:42:18.204973 24094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:18.200009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:54199 (local address 127.14.51.126:38307)
0806 06:42:18.200071 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.200073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.200156 (+ 83us) server_negotiation.cc:407] Connection header received
0806 06:42:18.200214 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.200215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.200224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.200266 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.200649 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.201590 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.202317 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.202505 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.202568 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.204928 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.51.66:54199: BlockingRecv error: recv got EOF from 127.14.51.66:54199 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:42:18.241837 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23597
I20260806 06:42:18.248047 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23755
I20260806 06:42:18.252779 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23894
I20260806 06:42:18.257270 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 23524
2026-08-06T06:42:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17514 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T06:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:18Z Disabled control of system clock
I20260806 06:42:18.284945 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35707
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:35707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:42:18.365424 24099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:18.365641 24099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:18.365664 24099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:18.366750 24099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:18.366786 24099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:18.366798 24099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:18.366811 24099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:18.366823 24099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:18.368182 24099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:35707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:18.368371 24099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:18.368685 24099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:18.373490 24104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.373518 24107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.373533 24105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.373569 24099 server_base.cc:1034] running on GCE node
I20260806 06:42:18.374065 24099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:18.374374 24099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:18.375533 24099 hybrid_clock.cc:648] HybridClock initialized: now 1785998538375513 us; error 39 us; skew 500 ppm
I20260806 06:42:18.376734 24099 webserver.cc:460] Webserver started at http://127.14.51.126:43697/ using document root <none> and password file <none>
I20260806 06:42:18.376963 24099 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:18.377029 24099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:18.377161 24099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:18.378191 24099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "8603e2ed0ad3493f90fdd1313acd3eb1"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "740d4d8f007d0f45d57aabdd8ae20c0d"
server_key_iv: "8ecac7f9cdf3654b814291b4af59e1c8"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.378537 24099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "8603e2ed0ad3493f90fdd1313acd3eb1"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "740d4d8f007d0f45d57aabdd8ae20c0d"
server_key_iv: "8ecac7f9cdf3654b814291b4af59e1c8"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.380007 24099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:42:18.380736 24113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.380918 24099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:18.381023 24099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "8603e2ed0ad3493f90fdd1313acd3eb1"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "740d4d8f007d0f45d57aabdd8ae20c0d"
server_key_iv: "8ecac7f9cdf3654b814291b4af59e1c8"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.381100 24099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:18.396366 24099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:18.396632 24099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:18.396772 24099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:18.400866 24099 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:35707
I20260806 06:42:18.400895 24168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:35707 every 8 connection(s)
I20260806 06:42:18.401214 24099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:42:18.401820 24169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:18.404297 24169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1: Bootstrap starting.
I20260806 06:42:18.404817 24169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:18.405061 24169 log.cc:824] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:18.405637 24169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1: No bootstrap required, opened a new log
I20260806 06:42:18.407058 24169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } }
I20260806 06:42:18.407240 24169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:18.407276 24169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8603e2ed0ad3493f90fdd1313acd3eb1, State: Initialized, Role: FOLLOWER
I20260806 06:42:18.407377 24169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } }
I20260806 06:42:18.407433 24169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:42:18.407452 24169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:42:18.407482 24169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:18.407927 24169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } }
I20260806 06:42:18.408006 24169 leader_election.cc:302] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8603e2ed0ad3493f90fdd1313acd3eb1; no voters:
I20260806 06:42:18.408141 24169 leader_election.cc:288] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:42:18.408175 24172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:42:18.408303 24172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 8603e2ed0ad3493f90fdd1313acd3eb1, State: Running, Role: LEADER
I20260806 06:42:18.408390 24169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:42:18.408461 24172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } }
I20260806 06:42:18.409019 24173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } } }
I20260806 06:42:18.409219 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24099
I20260806 06:42:18.409276 24173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:18.409307 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:42:18.409206 24174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8603e2ed0ad3493f90fdd1313acd3eb1. Latest consensus state: current_term: 1 leader_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8603e2ed0ad3493f90fdd1313acd3eb1" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35707 } } }
I20260806 06:42:18.409536 14541 external_mini_cluster.cc:1428] Setting key 5e2767a52a57256fff5081f7a0c82627
I20260806 06:42:18.409634 24174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:18.409643 24181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:42:18.410362 24181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:42:18.411995 24181 catalog_manager.cc:1347] Generated new cluster ID: 89035b8874f94db894cac1ba30013df8
I20260806 06:42:18.412045 24181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:42:18.415380 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.410079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.14.51.126:35707)
0806 06:42:18.410301 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.410321 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.410354 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:42:18.410468 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.410475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.410543 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.410691 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.410926 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.412237 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.413325 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.413906 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.414010 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.414701 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:18.414704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:18.414705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:18.414894 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:18.414932 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:18.414944 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:18.415002 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:18.415096 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:42:18.415183 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260806 06:42:18.420948 24181 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:42:18.421597 24181 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:42:18.431532 24181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8603e2ed0ad3493f90fdd1313acd3eb1: Generated new TSK 0
I20260806 06:42:18.431734 24181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:42:18.437083 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--builtin_ntp_servers=127.14.51.84:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:42:18.515877 24194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:18.516067 24194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:18.516086 24194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:18.517239 24194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:18.517279 24194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:18.517304 24194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:42:18.518607 24194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:18.518841 24194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:18.519174 24194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:18.524796 24200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.524844 24194 server_base.cc:1034] running on GCE node
W20260806 06:42:18.524945 24202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.524950 24199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.525166 24194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:18.525400 24194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:18.526561 24194 hybrid_clock.cc:648] HybridClock initialized: now 1785998538526544 us; error 34 us; skew 500 ppm
I20260806 06:42:18.527967 24194 webserver.cc:460] Webserver started at http://127.14.51.65:42259/ using document root <none> and password file <none>
I20260806 06:42:18.528210 24194 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:18.528280 24194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:18.528436 24194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:18.529434 24194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "26d13453d8de4dd5b311d9f48a77a59f"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "5a15eb3ba4b27a268f5398965c67fb69"
server_key_iv: "aede24573206842534ef164237c42de7"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.529786 24194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "26d13453d8de4dd5b311d9f48a77a59f"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "5a15eb3ba4b27a268f5398965c67fb69"
server_key_iv: "aede24573206842534ef164237c42de7"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.531097 24194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:42:18.531893 24208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.532075 24194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:18.532155 24194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "26d13453d8de4dd5b311d9f48a77a59f"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "5a15eb3ba4b27a268f5398965c67fb69"
server_key_iv: "aede24573206842534ef164237c42de7"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.532223 24194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:18.559095 24194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:18.559463 24194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:18.559625 24194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:18.559845 24194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:18.560180 24194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:18.560235 24194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.560276 24194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:18.560303 24194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.566115 24194 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34999
I20260806 06:42:18.566151 24333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34999 every 8 connection(s)
I20260806 06:42:18.566475 24194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:42:18.571696 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.567225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:44055 (local address 127.14.51.126:35707)
0806 06:42:18.567274 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.567276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.567493 (+ 217us) server_negotiation.cc:407] Connection header received
0806 06:42:18.568021 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.568023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.568038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.568104 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.568764 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.569278 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.570253 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.570469 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.570556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.571413 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:18.571415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:18.571415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:18.571420 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:18.571434 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:18.571438 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:18.571493 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:18.571585 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:42:18.571616 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:42:18.571906 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.567189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:35707 (local address 127.14.51.65:44055)
0806 06:42:18.567369 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:18.567401 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:18.567909 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:18.568173 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:18.568177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:18.568187 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:18.568661 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.568665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.569378 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.569379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.570152 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.570154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.570555 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.570556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.570692 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.571258 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:18.571258 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:18.571285 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:18.571306 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:18.571494 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:18.571499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:18.571502 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:42:18.571588 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:42:18.571684 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260806 06:42:18.572114 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24194
I20260806 06:42:18.572202 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:42:18.572381 24334 heartbeater.cc:347] Connected to a master server at 127.14.51.126:35707
I20260806 06:42:18.572476 24334 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:18.572489 14541 external_mini_cluster.cc:1428] Setting key 703fc1118e98500ca579b2bc764dd143
I20260806 06:42:18.572665 24334 heartbeater.cc:511] Master 127.14.51.126:35707 requested a full tablet report, sending...
I20260806 06:42:18.573239 24130 ts_manager.cc:194] Registered new tserver with Master: 26d13453d8de4dd5b311d9f48a77a59f (127.14.51.65:34999)
I20260806 06:42:18.573467 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--builtin_ntp_servers=127.14.51.84:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:42:18.573963 24130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.65:44055
W20260806 06:42:18.652280 24337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:18.652467 24337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:18.652488 24337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:18.653564 24337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:18.653599 24337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:18.653622 24337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:42:18.654893 24337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:18.655057 24337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:18.655296 24337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:18.660007 24344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.659981 24346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.660017 24343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.660249 24337 server_base.cc:1034] running on GCE node
I20260806 06:42:18.660445 24337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:18.660627 24337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:18.661791 24337 hybrid_clock.cc:648] HybridClock initialized: now 1785998538661771 us; error 30 us; skew 500 ppm
I20260806 06:42:18.663103 24337 webserver.cc:460] Webserver started at http://127.14.51.66:43351/ using document root <none> and password file <none>
I20260806 06:42:18.663359 24337 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:18.663427 24337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:18.663574 24337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:18.664599 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "f9a097b443cb48e4a2007b7d4deb90ed"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "34a788ebe97f691faf4c93a0260c5a1c"
server_key_iv: "485ffa9fd9c29828664a1bed2fb5d1e5"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.664964 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "f9a097b443cb48e4a2007b7d4deb90ed"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "34a788ebe97f691faf4c93a0260c5a1c"
server_key_iv: "485ffa9fd9c29828664a1bed2fb5d1e5"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.666424 24337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:42:18.667104 24352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.667295 24337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:18.667394 24337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "f9a097b443cb48e4a2007b7d4deb90ed"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "34a788ebe97f691faf4c93a0260c5a1c"
server_key_iv: "485ffa9fd9c29828664a1bed2fb5d1e5"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.667483 24337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:18.679224 24337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:18.679472 24337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:18.679616 24337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:18.679828 24337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:18.680140 24337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:18.680193 24337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.680226 24337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:18.680251 24337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.685967 24337 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:40331
I20260806 06:42:18.686015 24470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:40331 every 8 connection(s)
I20260806 06:42:18.686296 24337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:42:18.688421 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24337
I20260806 06:42:18.688519 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:42:18.688714 14541 external_mini_cluster.cc:1428] Setting key 1e8da2c1c35543358566b98a0c267036
I20260806 06:42:18.689683 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--builtin_ntp_servers=127.14.51.84:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:42:18.691669 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.686879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:45405 (local address 127.14.51.126:35707)
0806 06:42:18.686922 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.686924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.687184 (+ 260us) server_negotiation.cc:407] Connection header received
0806 06:42:18.687614 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.687615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.687624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.687671 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.688280 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.689098 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.690137 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.690459 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.690536 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.691416 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:18.691417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:18.691417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:18.691421 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:18.691435 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:18.691439 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:18.691486 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:18.691591 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:42:18.691614 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:42:18.691973 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.686881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:35707 (local address 127.14.51.66:45405)
0806 06:42:18.687041 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:18.687070 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:18.687531 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:18.687713 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:18.687716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:18.687723 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:18.688166 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.688170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.689195 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.689196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.690055 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.690057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.690543 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.690544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.690646 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.691259 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:18.691260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:18.691297 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:18.691323 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:18.691513 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:18.691516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:18.691519 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:42:18.691586 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:42:18.691688 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260806 06:42:18.692605 24471 heartbeater.cc:347] Connected to a master server at 127.14.51.126:35707
I20260806 06:42:18.692680 24471 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:18.692862 24471 heartbeater.cc:511] Master 127.14.51.126:35707 requested a full tablet report, sending...
I20260806 06:42:18.693239 24130 ts_manager.cc:194] Registered new tserver with Master: f9a097b443cb48e4a2007b7d4deb90ed (127.14.51.66:40331)
I20260806 06:42:18.693616 24130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.66:45405
W20260806 06:42:18.769505 24474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:18.769727 24474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:18.769848 24474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:18.771087 24474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:18.771166 24474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:18.771214 24474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:42:18.772528 24474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:35707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:18.772789 24474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:18.773082 24474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:18.777894 24480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.777951 24482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.777954 24479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.778163 24474 server_base.cc:1034] running on GCE node
I20260806 06:42:18.778426 24474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:18.778687 24474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:18.779875 24474 hybrid_clock.cc:648] HybridClock initialized: now 1785998538779861 us; error 34 us; skew 500 ppm
I20260806 06:42:18.781139 24474 webserver.cc:460] Webserver started at http://127.14.51.67:40415/ using document root <none> and password file <none>
I20260806 06:42:18.781378 24474 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:18.781424 24474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:18.781587 24474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:18.782588 24474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "a12b90ab51e24b03a306d5d27b06029c"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "01427258c2e2e59aebb4edd2f66f77f1"
server_key_iv: "6f492915d08df8ce3ed61648b18147df"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.782969 24474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "a12b90ab51e24b03a306d5d27b06029c"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "01427258c2e2e59aebb4edd2f66f77f1"
server_key_iv: "6f492915d08df8ce3ed61648b18147df"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.784423 24474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:42:18.785130 24488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.785336 24474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:18.785485 24474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "a12b90ab51e24b03a306d5d27b06029c"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "01427258c2e2e59aebb4edd2f66f77f1"
server_key_iv: "6f492915d08df8ce3ed61648b18147df"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.785561 24474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:18.796175 24474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:18.796429 24474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:18.796564 24474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:18.796761 24474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:18.797057 24474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:18.797093 24474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.797113 24474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:18.797153 24474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.802721 24474 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:33067
I20260806 06:42:18.802765 24607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:33067 every 8 connection(s)
I20260806 06:42:18.803042 24474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:42:18.805135 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24474
I20260806 06:42:18.805208 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:42:18.805388 14541 external_mini_cluster.cc:1428] Setting key 2b685872e8c8cfb0c19ec7f8dc455ddb
I20260806 06:42:18.808269 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.803695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:47101 (local address 127.14.51.126:35707)
0806 06:42:18.803784 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.803786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.804044 (+ 258us) server_negotiation.cc:407] Connection header received
0806 06:42:18.804516 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.804517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.804529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.804586 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.805326 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.806007 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.806907 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.807106 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.807207 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.808059 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:18.808060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:18.808061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:18.808065 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:18.808079 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:18.808083 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:18.808126 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:18.808191 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:42:18.808215 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:42:18.808504 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.803665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:35707 (local address 127.14.51.67:47101)
0806 06:42:18.803890 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:18.803940 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:18.804401 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:18.804644 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:18.804647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:18.804655 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:18.805164 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.805168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.806080 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.806081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.806808 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:18.806810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:18.807217 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:18.807218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:18.807350 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:18.807941 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:18.807941 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:18.807967 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:18.807988 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:18.808139 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:18.808143 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:18.808145 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:42:18.808194 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:42:18.808267 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260806 06:42:18.809073 24608 heartbeater.cc:347] Connected to a master server at 127.14.51.126:35707
I20260806 06:42:18.809170 24608 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:18.809335 24608 heartbeater.cc:511] Master 127.14.51.126:35707 requested a full tablet report, sending...
I20260806 06:42:18.809701 24130 ts_manager.cc:194] Registered new tserver with Master: a12b90ab51e24b03a306d5d27b06029c (127.14.51.67:33067)
I20260806 06:42:18.810128 24130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.67:47101
I20260806 06:42:18.816890 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:42:18.818571 24191 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:18.818116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40774 (local address 127.14.51.126:35707)
0806 06:42:18.818171 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.818173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.818284 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:42:18.818318 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.818318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.818328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.818375 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.818508 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40774: BlockingRecv error: recv got EOF from 127.0.0.1:40774 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:42:18.818608 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24194
I20260806 06:42:18.823791 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24337
I20260806 06:42:18.828501 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24474
I20260806 06:42:18.833263 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24099
2026-08-06T06:42:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (575 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T06:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:18Z Disabled control of system clock
I20260806 06:42:18.860080 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38525
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:42:18.939440 24619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:18.939625 24619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:18.939646 24619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:18.940675 24619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:18.940709 24619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:18.940722 24619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:18.940734 24619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:18.940747 24619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:18.941936 24619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:18.942106 24619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:18.942310 24619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:18.947002 24625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.947072 24626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:18.946995 24628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:18.947197 24619 server_base.cc:1034] running on GCE node
I20260806 06:42:18.947451 24619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:18.947715 24619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:18.948885 24619 hybrid_clock.cc:648] HybridClock initialized: now 1785998538948897 us; error 55 us; skew 500 ppm
I20260806 06:42:18.950054 24619 webserver.cc:460] Webserver started at http://127.14.51.126:38727/ using document root <none> and password file <none>
I20260806 06:42:18.950268 24619 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:18.950331 24619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:18.950488 24619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:18.951532 24619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "63885192e8884da6afeebf32397e908b"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "03dc9badbeba9d96c24055733b876d49"
server_key_iv: "3e54178c14eb8c8a8cdf286c56ba1cf1"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.951900 24619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "63885192e8884da6afeebf32397e908b"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "03dc9badbeba9d96c24055733b876d49"
server_key_iv: "3e54178c14eb8c8a8cdf286c56ba1cf1"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.953173 24619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:42:18.953913 24635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:18.954160 24619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:42:18.954268 24619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "63885192e8884da6afeebf32397e908b"
format_stamp: "Formatted at 2026-08-06 06:42:18 on dist-test-slave-1xrh"
server_key: "03dc9badbeba9d96c24055733b876d49"
server_key_iv: "3e54178c14eb8c8a8cdf286c56ba1cf1"
server_key_version: "encryptionkey@0"
I20260806 06:42:18.954349 24619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:18.957404 24619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:18.957640 24619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:18.957789 24619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:18.961357 24619 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38525
I20260806 06:42:18.961390 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38525 every 8 connection(s)
I20260806 06:42:18.961686 24619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:42:18.962297 24691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:18.963814 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24619
I20260806 06:42:18.963937 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:42:18.964139 14541 external_mini_cluster.cc:1428] Setting key 29f6b1879490b7bce86a7f5911ad4763
I20260806 06:42:18.965494 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b: Bootstrap starting.
I20260806 06:42:18.966188 24691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:18.966574 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:18.964658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50304 (local address 127.14.51.126:38525)
0806 06:42:18.964859 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:18.964866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:18.964888 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:42:18.965339 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:18.965345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:18.965381 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:18.965525 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:18.966167 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:18.966170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:18.966171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:18.966174 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:18.966222 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:18.966232 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:18.966322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:18.966358 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:42:18.966410 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260806 06:42:18.966574 24691 log.cc:824] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:18.967363 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b: No bootstrap required, opened a new log
I20260806 06:42:18.968658 24691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } }
I20260806 06:42:18.968786 24691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:18.968837 24691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63885192e8884da6afeebf32397e908b, State: Initialized, Role: FOLLOWER
I20260806 06:42:18.968959 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } }
I20260806 06:42:18.969029 24691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:42:18.969074 24691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:42:18.969121 24691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:18.969880 24691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } }
I20260806 06:42:18.969995 24691 leader_election.cc:302] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63885192e8884da6afeebf32397e908b; no voters:
I20260806 06:42:18.970181 24691 leader_election.cc:288] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:42:18.970244 24696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:42:18.970389 24696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [term 1 LEADER]: Becoming Leader. State: Replica: 63885192e8884da6afeebf32397e908b, State: Running, Role: LEADER
I20260806 06:42:18.970440 24691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:42:18.970479 24696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } }
I20260806 06:42:18.970829 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63885192e8884da6afeebf32397e908b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } } }
I20260806 06:42:18.970834 24698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63885192e8884da6afeebf32397e908b. Latest consensus state: current_term: 1 leader_uuid: "63885192e8884da6afeebf32397e908b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63885192e8884da6afeebf32397e908b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38525 } } }
I20260806 06:42:18.971006 24698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:18.970958 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:18.971501 24704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:42:18.972004 24704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:42:18.973519 24704 catalog_manager.cc:1347] Generated new cluster ID: 9841354cbec740c89258712951bd0c6b
I20260806 06:42:18.973579 24704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:42:18.988583 24704 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:42:18.988735 24704 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:42:18.992483 24704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63885192e8884da6afeebf32397e908b: Generated new TSK 0
I20260806 06:42:18.992651 24704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:42:18.993966 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--builtin_ntp_servers=127.14.51.84:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:42:19.086591 24716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:19.086899 24716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:19.086941 24716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:19.088191 24716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:19.088275 24716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:19.088322 24716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:42:19.089815 24716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:19.090078 24716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:19.090485 24716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:19.096426 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:19.096436 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:19.099890 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:19.103157 24716 server_base.cc:1034] running on GCE node
I20260806 06:42:19.103395 24716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:19.103667 24716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:19.104835 24716 hybrid_clock.cc:648] HybridClock initialized: now 1785998539104799 us; error 47 us; skew 500 ppm
I20260806 06:42:19.106415 24716 webserver.cc:460] Webserver started at http://127.14.51.65:39809/ using document root <none> and password file <none>
I20260806 06:42:19.106693 24716 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:19.106805 24716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:19.106968 24716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:19.108549 24716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "8030d4be95654b58b2f52cd8d7a8eea8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "262884d75a2fd90aa72c333cbde5f531"
server_key_iv: "15efd546a0b741b781f85f8325f36a20"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.109030 24716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "8030d4be95654b58b2f52cd8d7a8eea8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "262884d75a2fd90aa72c333cbde5f531"
server_key_iv: "15efd546a0b741b781f85f8325f36a20"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.110852 24716 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 06:42:19.111873 24732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.112052 24716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:42:19.112154 24716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "8030d4be95654b58b2f52cd8d7a8eea8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "262884d75a2fd90aa72c333cbde5f531"
server_key_iv: "15efd546a0b741b781f85f8325f36a20"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.112284 24716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:19.116432 24716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:19.116638 24716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:19.116730 24716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:19.116876 24716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:19.117117 24716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:19.117147 24716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.117167 24716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:19.117180 24716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.122494 24716 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:46607
I20260806 06:42:19.122571 24854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:46607 every 8 connection(s)
I20260806 06:42:19.122877 24716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:42:19.125547 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.123616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:56647 (local address 127.14.51.126:38525)
0806 06:42:19.123697 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:19.123699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:19.123888 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:42:19.124283 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:19.124284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:19.124289 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:19.124341 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:19.125350 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:19.125351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:19.125351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:19.125355 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:19.125369 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:19.125372 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:19.125419 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:19.125478 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:42:19.125490 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:42:19.125715 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.123577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38525 (local address 127.14.51.65:56647)
0806 06:42:19.123757 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:19.123788 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:19.124205 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:19.124387 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:19.124390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:19.124398 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:19.125221 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:19.125221 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:19.125247 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:19.125271 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:19.125418 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:19.125420 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:19.125422 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:42:19.125471 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:42:19.125513 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260806 06:42:19.126221 24856 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38525
I20260806 06:42:19.126317 24856 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:19.126472 24856 heartbeater.cc:511] Master 127.14.51.126:38525 requested a full tablet report, sending...
I20260806 06:42:19.126942 24653 ts_manager.cc:194] Registered new tserver with Master: 8030d4be95654b58b2f52cd8d7a8eea8 (127.14.51.65:46607)
I20260806 06:42:19.132227 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24716
I20260806 06:42:19.132323 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:42:19.132520 14541 external_mini_cluster.cc:1428] Setting key 0c02aefd7005f3208d06191697cfdf1b
I20260806 06:42:19.133414 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--builtin_ntp_servers=127.14.51.84:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:42:19.213304 24859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:19.213511 24859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:19.213534 24859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:19.214589 24859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:19.214623 24859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:19.214648 24859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:42:19.216025 24859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:19.216207 24859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:19.216420 24859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:19.221298 24864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:19.221308 24867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:19.221496 24859 server_base.cc:1034] running on GCE node
W20260806 06:42:19.221369 24865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:19.221796 24859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:19.222055 24859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:19.223246 24859 hybrid_clock.cc:648] HybridClock initialized: now 1785998539223210 us; error 56 us; skew 500 ppm
I20260806 06:42:19.224539 24859 webserver.cc:460] Webserver started at http://127.14.51.66:34279/ using document root <none> and password file <none>
I20260806 06:42:19.224797 24859 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:19.224865 24859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:19.225049 24859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:19.226100 24859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "5c72a6a099fc4da598ad28f995fcb0a8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "05810da6d422c883c6b87ca4cff49dd6"
server_key_iv: "02924b6d867a2a3798e78f585f8cca23"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.226454 24859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "5c72a6a099fc4da598ad28f995fcb0a8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "05810da6d422c883c6b87ca4cff49dd6"
server_key_iv: "02924b6d867a2a3798e78f585f8cca23"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.227768 24859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:42:19.228497 24874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.228685 24859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:42:19.228787 24859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "5c72a6a099fc4da598ad28f995fcb0a8"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "05810da6d422c883c6b87ca4cff49dd6"
server_key_iv: "02924b6d867a2a3798e78f585f8cca23"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.228879 24859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:19.232167 24859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:19.232410 24859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:19.232544 24859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:19.232761 24859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:19.233064 24859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:19.233119 24859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.233156 24859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:19.233184 24859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.238765 24859 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:37403
I20260806 06:42:19.238843 24993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:37403 every 8 connection(s)
I20260806 06:42:19.239166 24859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:42:19.242033 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.239865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:35383 (local address 127.14.51.126:38525)
0806 06:42:19.239938 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:19.239940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:19.240099 (+ 159us) server_negotiation.cc:407] Connection header received
0806 06:42:19.240517 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:19.240519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:19.240524 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:19.240583 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:19.241783 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:19.241785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:19.241785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:19.241789 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:19.241803 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:19.241808 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:19.241867 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:19.241965 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:42:19.241979 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:42:19.242192 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.239806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38525 (local address 127.14.51.66:35383)
0806 06:42:19.239978 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:19.240020 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:19.240432 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:19.240627 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:19.240631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:19.240638 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:19.241524 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:19.241524 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:19.241549 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:19.241573 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:19.241877 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:19.241881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:19.241883 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:42:19.241954 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:42:19.241998 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260806 06:42:19.242692 24994 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38525
I20260806 06:42:19.242791 24994 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:19.242970 24994 heartbeater.cc:511] Master 127.14.51.126:38525 requested a full tablet report, sending...
I20260806 06:42:19.243355 24653 ts_manager.cc:194] Registered new tserver with Master: 5c72a6a099fc4da598ad28f995fcb0a8 (127.14.51.66:37403)
I20260806 06:42:19.248310 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24859
I20260806 06:42:19.248427 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:42:19.248646 14541 external_mini_cluster.cc:1428] Setting key 2fab278cfe08e2a9ec92568ee5deb7fc
I20260806 06:42:19.249527 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--builtin_ntp_servers=127.14.51.84:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:42:19.332438 24997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:19.332639 24997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:19.332662 24997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:19.333761 24997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:19.333796 24997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:19.333858 24997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:42:19.335098 24997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:19.335331 24997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:19.335556 24997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:19.340257 25002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:19.340308 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:19.340337 25003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:19.340793 24997 server_base.cc:1034] running on GCE node
I20260806 06:42:19.341073 24997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:19.341315 24997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:19.342469 24997 hybrid_clock.cc:648] HybridClock initialized: now 1785998539342446 us; error 43 us; skew 500 ppm
I20260806 06:42:19.343991 24997 webserver.cc:460] Webserver started at http://127.14.51.67:34081/ using document root <none> and password file <none>
I20260806 06:42:19.344233 24997 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:19.344278 24997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:19.344424 24997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:19.345472 24997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "63b7f3ada0cb440aadfd2d314edea704"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "421d21265d51b5aa496ba7ef4ae6b02e"
server_key_iv: "b816a21c96a140abd5d36a3d856384cb"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.345846 24997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "63b7f3ada0cb440aadfd2d314edea704"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "421d21265d51b5aa496ba7ef4ae6b02e"
server_key_iv: "b816a21c96a140abd5d36a3d856384cb"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.347178 24997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:42:19.347975 25012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.348165 24997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:19.348251 24997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "63b7f3ada0cb440aadfd2d314edea704"
format_stamp: "Formatted at 2026-08-06 06:42:19 on dist-test-slave-1xrh"
server_key: "421d21265d51b5aa496ba7ef4ae6b02e"
server_key_iv: "b816a21c96a140abd5d36a3d856384cb"
server_key_version: "encryptionkey@0"
I20260806 06:42:19.348323 24997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:19.351713 24997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:19.351946 24997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:19.352077 24997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:19.352267 24997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:19.352550 24997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:19.352582 24997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.352620 24997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:19.352689 24997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:19.358687 24997 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:35193
I20260806 06:42:19.358763 25128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:35193 every 8 connection(s)
I20260806 06:42:19.359067 24997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:42:19.362175 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.359841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:53177 (local address 127.14.51.126:38525)
0806 06:42:19.359921 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:19.359923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:19.360130 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:42:19.360618 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:19.360620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:19.360626 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:19.360695 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:19.361900 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:19.361902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:19.361902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:42:19.361907 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:19.361922 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:42:19.361926 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:19.361980 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:19.362088 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:42:19.362105 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:42:19.362360 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:19.359811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:38525 (local address 127.14.51.67:53177)
0806 06:42:19.360009 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:19.360040 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:19.360509 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:19.360757 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:19.360761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:19.360770 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:19.361741 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:42:19.361742 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:19.361768 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:42:19.361794 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:19.361987 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:19.361992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:19.361995 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:42:19.362084 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:42:19.362144 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260806 06:42:19.362826 25129 heartbeater.cc:347] Connected to a master server at 127.14.51.126:38525
I20260806 06:42:19.362897 25129 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:19.363016 25129 heartbeater.cc:511] Master 127.14.51.126:38525 requested a full tablet report, sending...
I20260806 06:42:19.363423 24653 ts_manager.cc:194] Registered new tserver with Master: 63b7f3ada0cb440aadfd2d314edea704 (127.14.51.67:35193)
I20260806 06:42:19.364151 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 24997
I20260806 06:42:19.364251 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:42:19.364432 14541 external_mini_cluster.cc:1428] Setting key 68370b0c777b9f8063418dc560cc9a04
I20260806 06:42:19.365335 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:42:19.367012 24694 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:19.366608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.14.51.126:38525)
0806 06:42:19.366663 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:19.366664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:19.366752 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:42:19.366801 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:19.366802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:19.366805 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:19.366848 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:19.366943 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35022: BlockingRecv error: recv got EOF from 127.0.0.1:35022 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:42:19.367154 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24716
I20260806 06:42:19.376667 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24859
I20260806 06:42:19.381187 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24997
I20260806 06:42:19.386185 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 24619
2026-08-06T06:42:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (554 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:42:19 dist-test-slave-1xrh krb5kdc[25140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:42:19 dist-test-slave-1xrh krb5kdc[25140](info): set up 2 sockets
Aug 06 06:42:19 dist-test-slave-1xrh krb5kdc[25140](info): commencing operation
krb5kdc: starting...
W20260806 06:42:20.476297 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:42:20 dist-test-slave-1xrh krb5kdc[25140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:42:20.595882 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:41919
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:20.674588 25153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:20.674806 25153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:20.674861 25153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:20.675949 25153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:42:20.676012 25153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:20.676035 25153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:20.676056 25153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:42:20.676084 25153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:42:20.677492 25153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:20.677762 25153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:20.677987 25153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:20.682575 25159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:20.682579 25161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:20.682632 25153 server_base.cc:1034] running on GCE node
W20260806 06:42:20.682580 25158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:20.683104 25153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:20.683477 25153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:20.684616 25153 hybrid_clock.cc:648] HybridClock initialized: now 1785998540684598 us; error 39 us; skew 500 ppm
Aug 06 06:42:20 dist-test-slave-1xrh krb5kdc[25140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998540, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:20.686985 25153 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:42:20.687395 25153 webserver.cc:460] Webserver started at http://127.14.51.126:39749/ using document root <none> and password file <none>
I20260806 06:42:20.687559 25153 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:20.687594 25153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:20.687672 25153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:20.688534 25153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "473fb4ff1c694f96aa2ce48a2a28fd08"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "06014495adbf335d442e9795f1280411"
server_key_iv: "851921aa8b9413c6038fdb40a7c6690d"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.688792 25153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "473fb4ff1c694f96aa2ce48a2a28fd08"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "06014495adbf335d442e9795f1280411"
server_key_iv: "851921aa8b9413c6038fdb40a7c6690d"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.690069 25153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:42:20.690732 25168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:20.690949 25153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:20.691040 25153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "473fb4ff1c694f96aa2ce48a2a28fd08"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "06014495adbf335d442e9795f1280411"
server_key_iv: "851921aa8b9413c6038fdb40a7c6690d"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.691111 25153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:20.704566 25153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:20.706161 25153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:20.706312 25153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:20.710119 25153 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:41919
I20260806 06:42:20.710150 25224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:41919 every 8 connection(s)
I20260806 06:42:20.710466 25153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:42:20.711416 25225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:20.713526 25225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: Bootstrap starting.
I20260806 06:42:20.714095 25225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:20.714387 25225 log.cc:824] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:20.715056 25225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: No bootstrap required, opened a new log
I20260806 06:42:20.716563 25225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } }
I20260806 06:42:20.716689 25225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:20.716710 25225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473fb4ff1c694f96aa2ce48a2a28fd08, State: Initialized, Role: FOLLOWER
I20260806 06:42:20.716871 25225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } }
I20260806 06:42:20.716933 25225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:42:20.716953 25225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:42:20.717020 25225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:20.717770 25225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } }
I20260806 06:42:20.717877 25225 leader_election.cc:302] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 473fb4ff1c694f96aa2ce48a2a28fd08; no voters:
I20260806 06:42:20.718060 25225 leader_election.cc:288] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:42:20.718119 25228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:42:20.718282 25228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [term 1 LEADER]: Becoming Leader. State: Replica: 473fb4ff1c694f96aa2ce48a2a28fd08, State: Running, Role: LEADER
I20260806 06:42:20.718312 25225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:42:20.718366 25228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } }
I20260806 06:42:20.718690 25230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473fb4ff1c694f96aa2ce48a2a28fd08. Latest consensus state: current_term: 1 leader_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } } }
I20260806 06:42:20.718684 25229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fb4ff1c694f96aa2ce48a2a28fd08" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 41919 } } }
I20260806 06:42:20.718783 25230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:20.718788 25229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08 [sys.catalog]: This master's current role is: LEADER
I20260806 06:42:20.719177 25237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:42:20.719728 25237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:42:20.719977 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 25153
I20260806 06:42:20.720077 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:42:20.720256 14541 external_mini_cluster.cc:1428] Setting key 2c2b6ebf879519776e04bdbfdb022e3b
I20260806 06:42:20.721792 25237 catalog_manager.cc:1347] Generated new cluster ID: f11ab2daf3694b9a9e84481cb3f0e990
I20260806 06:42:20.721846 25237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:42:20 dist-test-slave-1xrh krb5kdc[25140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:20.729888 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:20.720729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.14.51.126:41919)
0806 06:42:20.720923 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:20.720932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:20.720964 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:42:20.721083 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:20.721089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:20.721145 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:20.721285 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:20.722049 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.722948 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.723763 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.724216 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.724337 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:20.726668 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:20.726672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:20.726672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:20.726684 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:20.728325 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:20.728720 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:20.728722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:20.728723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:20.728823 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:20.729069 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:20.729070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:20.729071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:20.729290 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:20.729468 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:20.729543 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:42:20.729684 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260806 06:42:20.738772 25237 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:42:20.739362 25237 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:42:20.744200 25237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: Generated new TSK 0
I20260806 06:42:20.744369 25237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:42:20.773993 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:20.874325 25252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:20.874544 25252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:20.874583 25252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:20.875742 25252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:20.875821 25252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:20.875867 25252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:42:20.877333 25252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:20.877586 25252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:20.877825 25252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:20.882875 25258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:20.883033 25252 server_base.cc:1034] running on GCE node
W20260806 06:42:20.882905 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:20.882870 25257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:20.883381 25252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:20.883567 25252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:20.884773 25252 hybrid_clock.cc:648] HybridClock initialized: now 1785998540884757 us; error 34 us; skew 500 ppm
Aug 06 06:42:20 dist-test-slave-1xrh krb5kdc[25140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998540, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:20.887253 25252 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:42:20.887672 25252 webserver.cc:460] Webserver started at http://127.14.51.65:38085/ using document root <none> and password file <none>
I20260806 06:42:20.887897 25252 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:20.887964 25252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:20.888095 25252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:20.889150 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "fed8f3ac405248548ba216bfd2e08bd4"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "792550e22a8212d754f09f060f8c8ef0"
server_key_iv: "a64a7c6ed840e37d6d760409e8f9a2c5"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.889518 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "fed8f3ac405248548ba216bfd2e08bd4"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "792550e22a8212d754f09f060f8c8ef0"
server_key_iv: "a64a7c6ed840e37d6d760409e8f9a2c5"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.890782 25252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:42:20.891585 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:20.891746 25252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:20.891887 25252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "fed8f3ac405248548ba216bfd2e08bd4"
format_stamp: "Formatted at 2026-08-06 06:42:20 on dist-test-slave-1xrh"
server_key: "792550e22a8212d754f09f060f8c8ef0"
server_key_iv: "a64a7c6ed840e37d6d760409e8f9a2c5"
server_key_version: "encryptionkey@0"
I20260806 06:42:20.891961 25252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:20.907678 25252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:20.909370 25252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:20.909526 25252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:20.909744 25252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:20.910033 25252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:20.910089 25252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:20.910122 25252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:20.910185 25252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:20.915776 25252 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34913
I20260806 06:42:20.915810 25385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34913 every 8 connection(s)
I20260806 06:42:20.916121 25252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:42:20.918994 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 25252
I20260806 06:42:20.919098 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:42:20.919303 14541 external_mini_cluster.cc:1428] Setting key 530f7ac800a838fd7edab52c25a6a4da
Aug 06 06:42:20 dist-test-slave-1xrh krb5kdc[25140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998540, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:20.926014 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:20.916758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:48373 (local address 127.14.51.126:41919)
0806 06:42:20.916846 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:20.916848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:20.917088 (+ 240us) server_negotiation.cc:407] Connection header received
0806 06:42:20.917505 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:20.917506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:20.917520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:20.917569 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:20.918360 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.919019 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.920597 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.920783 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.920835 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:20.923214 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:20.923215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:20.923216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:20.923223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:20.924647 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:20.925082 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:20.925084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:20.925084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:20.925139 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:20.925390 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:20.925391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:20.925392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:20.925506 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:20.925573 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:20.925834 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 06:42:20.925928 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:42:20.926200 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:20.916719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41919 (local address 127.14.51.65:48373)
0806 06:42:20.916943 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:20.916989 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:20.917426 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:20.917614 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:20.917617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:20.918121 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:20.918286 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:20.918290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.919118 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.919169 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:20.920504 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:20.920507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:20.920894 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:20.920895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:20.921040 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:20.921510 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:20.921520 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:20.923072 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:20.924749 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:20.924756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:20.924758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:20.924981 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:20.925211 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:20.925212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:20.925213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:20.925317 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:20.925576 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:20.925582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:20.925706 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 06:42:20.925801 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:42:20.925944 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":120,"threads_started":1}
I20260806 06:42:20.926594 25386 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41919
I20260806 06:42:20.926676 25386 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:20.926862 25386 heartbeater.cc:511] Master 127.14.51.126:41919 requested a full tablet report, sending...
I20260806 06:42:20.927349 25185 ts_manager.cc:194] Registered new tserver with Master: fed8f3ac405248548ba216bfd2e08bd4 (127.14.51.65:34913)
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
I20260806 06:42:20.927944 25185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:48373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:42:20.944286 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:21.023244 25391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:21.023463 25391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:21.023485 25391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:21.024551 25391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:21.024590 25391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:21.024616 25391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:42:21.025938 25391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:21.026119 25391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:21.026281 25391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:21.030967 25400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:21.031078 25391 server_base.cc:1034] running on GCE node
W20260806 06:42:21.030982 25396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:21.030982 25398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:21.031419 25391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:21.031649 25391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:21.032809 25391 hybrid_clock.cc:648] HybridClock initialized: now 1785998541032804 us; error 35 us; skew 500 ppm
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998541, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:21.035307 25391 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:42:21.035697 25391 webserver.cc:460] Webserver started at http://127.14.51.66:34547/ using document root <none> and password file <none>
I20260806 06:42:21.035920 25391 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:21.035985 25391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:21.036118 25391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:21.037098 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "658cb74a815b4006ba57a7c8344125a4"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "7fdbb2c991e6a3954c9b0814f15bc66c"
server_key_iv: "1c1ca73dbc892ee9f1b191ac70e1498d"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.037424 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "658cb74a815b4006ba57a7c8344125a4"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "7fdbb2c991e6a3954c9b0814f15bc66c"
server_key_iv: "1c1ca73dbc892ee9f1b191ac70e1498d"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.038635 25391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:42:21.039393 25407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.039553 25391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:21.039647 25391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "658cb74a815b4006ba57a7c8344125a4"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "7fdbb2c991e6a3954c9b0814f15bc66c"
server_key_iv: "1c1ca73dbc892ee9f1b191ac70e1498d"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.039724 25391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:21.067564 25391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:21.069869 25391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:21.069988 25391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:21.070181 25391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:21.070488 25391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:21.070547 25391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.070582 25391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:21.070634 25391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.076939 25391 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:37851
I20260806 06:42:21.076982 25524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:37851 every 8 connection(s)
I20260806 06:42:21.077280 25391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:42:21.078691 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 25391
I20260806 06:42:21.078806 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:42:21.078999 14541 external_mini_cluster.cc:1428] Setting key 55f198e3bbcc89bf66b1223edb71ec46
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998541, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:21.088001 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.077956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:54025 (local address 127.14.51.126:41919)
0806 06:42:21.078002 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.078004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.078194 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:42:21.078719 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.078721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.078735 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.078782 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:21.079993 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.080480 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.081873 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.082156 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.082234 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.084990 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:21.084992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:21.084993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:21.085002 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:21.086626 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.086996 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.086998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.086998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.087074 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.087346 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.087347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.087347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.087491 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:21.087566 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.087789 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.087882 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:42:21.088163 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.077920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41919 (local address 127.14.51.66:54025)
0806 06:42:21.078061 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.078109 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.078610 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.078858 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.078862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.079630 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:21.079901 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.079907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.080572 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.080573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.081762 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.081765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.082270 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.082271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.082409 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.082948 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:21.082959 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:21.084887 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:21.086711 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:21.086716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:21.086717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:21.086922 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:21.087175 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:21.087176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:21.087176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:21.087279 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:21.087558 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:21.087561 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:21.087661 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.087736 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.087926 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260806 06:42:21.088660 25525 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41919
I20260806 06:42:21.088734 25525 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:42:21.088925 25525 heartbeater.cc:511] Master 127.14.51.126:41919 requested a full tablet report, sending...
I20260806 06:42:21.089309 25185 ts_manager.cc:194] Registered new tserver with Master: 658cb74a815b4006ba57a7c8344125a4 (127.14.51.66:37851)
I20260806 06:42:21.089671 25185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:54025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:42:21.106040 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:42:21.185770 25530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:42:21.185984 25530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:42:21.186007 25530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:42:21.187163 25530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:42:21.187232 25530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:42:21.187276 25530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:42:21.188869 25530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:42:21.189059 25530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:42:21.189216 25530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:42:21.194137 25538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:42:21.194175 25535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:21.194393 25530 server_base.cc:1034] running on GCE node
W20260806 06:42:21.194175 25536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:42:21.194638 25530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:42:21.194885 25530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:42:21.196022 25530 hybrid_clock.cc:648] HybridClock initialized: now 1785998541196016 us; error 30 us; skew 500 ppm
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998541, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:42:21.198297 25530 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:42:21.198684 25530 webserver.cc:460] Webserver started at http://127.14.51.67:33051/ using document root <none> and password file <none>
I20260806 06:42:21.198915 25530 fs_manager.cc:359] Metadata directory not provided
I20260806 06:42:21.198998 25530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:42:21.199199 25530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:42:21.200191 25530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "3f19a8013f394e7dbd7d4cd42c14b903"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "d3f52b050bbaff6fedbf10ba67df1fe5"
server_key_iv: "4ee8762d03f83d208ad2d4e7fdbf5894"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.200547 25530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "3f19a8013f394e7dbd7d4cd42c14b903"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "d3f52b050bbaff6fedbf10ba67df1fe5"
server_key_iv: "4ee8762d03f83d208ad2d4e7fdbf5894"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.201795 25530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:42:21.202456 25545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.202623 25530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:42:21.202709 25530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "3f19a8013f394e7dbd7d4cd42c14b903"
format_stamp: "Formatted at 2026-08-06 06:42:21 on dist-test-slave-1xrh"
server_key: "d3f52b050bbaff6fedbf10ba67df1fe5"
server_key_iv: "4ee8762d03f83d208ad2d4e7fdbf5894"
server_key_version: "encryptionkey@0"
I20260806 06:42:21.202790 25530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:42:21.222368 25530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:42:21.224112 25530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:42:21.224260 25530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:42:21.224474 25530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:42:21.224762 25530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:42:21.224817 25530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.224856 25530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:42:21.224882 25530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:42:21.231181 25530 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:43571
I20260806 06:42:21.231230 25661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:43571 every 8 connection(s)
I20260806 06:42:21.231527 25530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998541, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:42:21.239758 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.232190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:53863 (local address 127.14.51.126:41919)
0806 06:42:21.232231 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.232233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.232405 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:42:21.232836 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.232838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.232847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.232901 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:21.233732 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.234188 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.235060 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.235308 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.235363 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.237392 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:21.237394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:21.237394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:21.237399 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:21.238562 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.238895 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.238895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.238895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.238945 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.239224 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.239224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.239225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.239321 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:21.239373 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.239628 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.239696 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:42:21.239820 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.232147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:41919 (local address 127.14.51.67:53863)
0806 06:42:21.232296 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.232333 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.232759 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.232963 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.232966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.233490 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:42:21.233663 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.233667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.234261 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.234261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.234983 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.234985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.235377 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.235378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.235492 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.235995 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:42:21.236003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:42:21.237316 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:42:21.238629 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:21.238634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:21.238636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:21.238833 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:21.239020 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:42:21.239020 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:42:21.239021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:42:21.239118 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:42:21.239378 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:42:21.239382 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:42:21.239485 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.239568 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.239676 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260806 06:42:21.240305 25662 heartbeater.cc:347] Connected to a master server at 127.14.51.126:41919
I20260806 06:42:21.240381 25662 heartbeater.cc:464] Registering TS with master...
I20260806 06:42:21.240600 25662 heartbeater.cc:511] Master 127.14.51.126:41919 requested a full tablet report, sending...
I20260806 06:42:21.240655 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 25530
I20260806 06:42:21.240746 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:42:21.240945 14541 external_mini_cluster.cc:1428] Setting key f9df012f2190d545c7953a904df535cf
I20260806 06:42:21.241007 25185 ts_manager.cc:194] Registered new tserver with Master: 3f19a8013f394e7dbd7d4cd42c14b903 (127.14.51.67:43571)
I20260806 06:42:21.241438 25185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:53863
I20260806 06:42:21.241844 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:42:21.248868 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.243151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.14.51.126:41919)
0806 06:42:21.243202 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.243204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.243281 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:42:21.243345 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.243346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.243355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.243401 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:42:21.244002 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.244613 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.245307 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.245484 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.245529 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.246544 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:42:21.246546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:42:21.246546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:42:21.246553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:42:21.247965 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.248258 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.248258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.248259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.248306 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:42:21.248471 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:42:21.248471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:42:21.248471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:42:21.248607 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:42:21.248677 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.248761 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.248813 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:42:21.250447 25184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:42:21.252106 25184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:42:21.258121 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:37851 (local address 127.0.0.1:57972)
0806 06:42:21.254413 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.254450 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.254510 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.255028 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.255029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.255031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:42:21.255079 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.255081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.256361 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.256362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.257143 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.257145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.257843 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.257844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.257915 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.257928 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.258001 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.258049 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":52,"threads_started":1}
I20260806 06:42:21.258168 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.14.51.66:37851)
0806 06:42:21.254802 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.254807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.254817 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:42:21.254854 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.254856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.254870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.254945 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:42:21.255192 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.256250 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.257226 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.257764 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.257835 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.257888 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.258023 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.258072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260806 06:42:21.258582 25673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34913 (local address 127.0.0.1:57494)
0806 06:42:21.254425 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.254451 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.254511 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.254852 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.254855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.254857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:42:21.254919 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.254921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.256298 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.256299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.257380 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.257383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.258352 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.258354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.258413 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.258417 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.258472 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.258509 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260806 06:42:21.258706 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.14.51.65:34913)
0806 06:42:21.254605 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.254610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.254619 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:21.254662 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.254663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.254675 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.254763 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:42:21.255031 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.256191 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.257470 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.258170 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.258246 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.258312 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.258509 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.258568 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260806 06:42:21.258940 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:43571 (local address 127.0.0.1:47422)
0806 06:42:21.254857 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.254868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.254921 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.255238 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.255238 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.255241 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:42:21.255282 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.255283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.256105 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.256106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.257593 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.257595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.258680 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.258681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.258749 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.258760 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.258813 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.258859 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":68,"threads_started":1}
I20260806 06:42:21.259266 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.254834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.14.51.67:43571)
0806 06:42:21.255012 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.255016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.255025 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:42:21.255058 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.255060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.255075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.255181 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:42:21.255361 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.255987 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.257675 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.258421 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.258511 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.259105 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.259163 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.259201 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260806 06:42:21.259251 25456 tablet_service.cc:1467] Processing CreateTablet for tablet e6964580d5f9454984be5b270cdd29fa (DEFAULT_TABLE table=test-table [id=c1acb1d111c1473793478ad8f7c8db84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:42:21.259433 25316 tablet_service.cc:1467] Processing CreateTablet for tablet e6964580d5f9454984be5b270cdd29fa (DEFAULT_TABLE table=test-table [id=c1acb1d111c1473793478ad8f7c8db84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:42:21.259626 25456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6964580d5f9454984be5b270cdd29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:21.259716 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6964580d5f9454984be5b270cdd29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:21.262005 25679 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: Bootstrap starting.
I20260806 06:42:21.262243 25681 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Bootstrap starting.
I20260806 06:42:21.262676 25679 tablet_bootstrap.cc:654] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:21.262642 25595 tablet_service.cc:1467] Processing CreateTablet for tablet e6964580d5f9454984be5b270cdd29fa (DEFAULT_TABLE table=test-table [id=c1acb1d111c1473793478ad8f7c8db84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:42:21.262758 25681 tablet_bootstrap.cc:654] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:21.263003 25595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6964580d5f9454984be5b270cdd29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:42:21.263293 25681 log.cc:824] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:21.263314 25679 log.cc:824] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:21.264029 25679 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: No bootstrap required, opened a new log
I20260806 06:42:21.264029 25681 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: No bootstrap required, opened a new log
I20260806 06:42:21.264104 25681 ts_tablet_manager.cc:1397] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:42:21.264104 25679 ts_tablet_manager.cc:1397] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:42:21.265750 25684 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Bootstrap starting.
I20260806 06:42:21.266029 25679 raft_consensus.cc:348] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.266016 25681 raft_consensus.cc:348] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.266176 25679 raft_consensus.cc:374] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:21.266176 25681 raft_consensus.cc:374] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:21.266209 25679 raft_consensus.cc:729] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed8f3ac405248548ba216bfd2e08bd4, State: Initialized, Role: FOLLOWER
I20260806 06:42:21.266209 25681 raft_consensus.cc:729] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658cb74a815b4006ba57a7c8344125a4, State: Initialized, Role: FOLLOWER
I20260806 06:42:21.266278 25684 tablet_bootstrap.cc:654] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Neither blocks nor log segments found. Creating new log.
I20260806 06:42:21.266314 25681 consensus_queue.cc:260] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.266314 25679 consensus_queue.cc:260] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.266592 25681 ts_tablet_manager.cc:1428] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:42:21.266592 25679 ts_tablet_manager.cc:1428] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:42:21.266629 25386 heartbeater.cc:503] Master 127.14.51.126:41919 was elected leader, sending a full tablet report...
I20260806 06:42:21.266700 25525 heartbeater.cc:503] Master 127.14.51.126:41919 was elected leader, sending a full tablet report...
I20260806 06:42:21.266866 25684 log.cc:824] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Log is configured to *not* fsync() on all Append() calls
I20260806 06:42:21.267732 25684 tablet_bootstrap.cc:492] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: No bootstrap required, opened a new log
I20260806 06:42:21.267832 25684 ts_tablet_manager.cc:1397] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:42:21.269142 25684 raft_consensus.cc:348] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.269260 25684 raft_consensus.cc:374] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:42:21.269281 25684 raft_consensus.cc:729] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f19a8013f394e7dbd7d4cd42c14b903, State: Initialized, Role: FOLLOWER
I20260806 06:42:21.269385 25684 consensus_queue.cc:260] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.269609 25684 ts_tablet_manager.cc:1428] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:42:21.269662 25662 heartbeater.cc:503] Master 127.14.51.126:41919 was elected leader, sending a full tablet report...
W20260806 06:42:21.327827 25526 tablet.cc:2369] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:42:21.372403 25686 raft_consensus.cc:482] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:42:21.372568 25686 raft_consensus.cc:504] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.372963 25686 leader_election.cc:288] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fed8f3ac405248548ba216bfd2e08bd4 (127.14.51.65:34913), 3f19a8013f394e7dbd7d4cd42c14b903 (127.14.51.67:43571)
I20260806 06:42:21.375823 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.373270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:35911 (local address 127.14.51.65:34913)
0806 06:42:21.373317 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.373319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.373335 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:42:21.373378 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.373378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.373386 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.373452 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:42:21.373613 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.374099 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375034 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.375447 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375540 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.375557 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.375720 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.375754 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:42:21.375823 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.373184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34913 (local address 127.14.51.66:35911)
0806 06:42:21.373256 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.373268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.373314 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.373462 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.373464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.373466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:42:21.373545 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.373547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.374190 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.374191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.374934 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.374936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.375537 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.375631 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.375634 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.375710 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.375751 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":5}
I20260806 06:42:21.376101 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.373312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:43571 (local address 127.14.51.66:34125)
0806 06:42:21.373510 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.373519 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:42:21.373570 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:42:21.373749 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:42:21.373750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:42:21.373751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:42:21.373836 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.373837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.374458 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.374459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.375288 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:42:21.375291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.375831 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:42:21.375918 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.375924 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:42:21.375990 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:42:21.376030 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":130,"threads_started":1}
I20260806 06:42:21.376109 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.373482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:34125 (local address 127.14.51.67:43571)
0806 06:42:21.373539 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.373555 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.373565 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:42:21.373663 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.373664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.373671 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.373743 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:42:21.373916 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.374371 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375379 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.375737 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.375830 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.375838 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.375995 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.376042 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:42:21.376174 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6964580d5f9454984be5b270cdd29fa" candidate_uuid: "658cb74a815b4006ba57a7c8344125a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed8f3ac405248548ba216bfd2e08bd4" is_pre_election: true
I20260806 06:42:21.376302 25337 raft_consensus.cc:2454] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658cb74a815b4006ba57a7c8344125a4 in term 0.
I20260806 06:42:21.376540 25411 leader_election.cc:302] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658cb74a815b4006ba57a7c8344125a4, fed8f3ac405248548ba216bfd2e08bd4; no voters:
I20260806 06:42:21.376696 25686 raft_consensus.cc:2781] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:42:21.376744 25686 raft_consensus.cc:482] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:42:21.376674 25615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6964580d5f9454984be5b270cdd29fa" candidate_uuid: "658cb74a815b4006ba57a7c8344125a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" is_pre_election: true
I20260806 06:42:21.376766 25686 raft_consensus.cc:3037] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:21.376817 25615 raft_consensus.cc:2454] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658cb74a815b4006ba57a7c8344125a4 in term 0.
I20260806 06:42:21.377563 25686 raft_consensus.cc:504] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.377691 25686 leader_election.cc:288] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [CANDIDATE]: Term 1 election: Requested vote from peers fed8f3ac405248548ba216bfd2e08bd4 (127.14.51.65:34913), 3f19a8013f394e7dbd7d4cd42c14b903 (127.14.51.67:43571)
I20260806 06:42:21.377820 25337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6964580d5f9454984be5b270cdd29fa" candidate_uuid: "658cb74a815b4006ba57a7c8344125a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed8f3ac405248548ba216bfd2e08bd4"
I20260806 06:42:21.377872 25615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6964580d5f9454984be5b270cdd29fa" candidate_uuid: "658cb74a815b4006ba57a7c8344125a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f19a8013f394e7dbd7d4cd42c14b903"
I20260806 06:42:21.377928 25337 raft_consensus.cc:3037] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:21.377956 25615 raft_consensus.cc:3037] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:42:21.378652 25615 raft_consensus.cc:2454] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658cb74a815b4006ba57a7c8344125a4 in term 1.
I20260806 06:42:21.378679 25337 raft_consensus.cc:2454] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658cb74a815b4006ba57a7c8344125a4 in term 1.
I20260806 06:42:21.378837 25411 leader_election.cc:302] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f19a8013f394e7dbd7d4cd42c14b903, 658cb74a815b4006ba57a7c8344125a4; no voters:
I20260806 06:42:21.378974 25686 raft_consensus.cc:2781] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:42:21.379241 25686 raft_consensus.cc:686] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 1 LEADER]: Becoming Leader. State: Replica: 658cb74a815b4006ba57a7c8344125a4, State: Running, Role: LEADER
I20260806 06:42:21.379352 25686 consensus_queue.cc:237] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } }
I20260806 06:42:21.380136 25185 catalog_manager.cc:5495] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658cb74a815b4006ba57a7c8344125a4 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "658cb74a815b4006ba57a7c8344125a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658cb74a815b4006ba57a7c8344125a4" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 37851 } health_report { overall_health: HEALTHY } } }
I20260806 06:42:21.401863 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:42:21.399776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.14.51.66:37851)
0806 06:42:21.399834 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:42:21.399836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:42:21.399844 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:42:21.399876 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:42:21.399877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:42:21.399893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:42:21.399954 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:42:21.400067 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.400511 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.401241 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:42:21.401416 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:42:21.401465 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:42:21.401597 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:42:21.401688 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:42:21.401744 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:42:21.401776 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:42:21.401791 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:42:21.403891 25337 raft_consensus.cc:1261] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 1 FOLLOWER]: Refusing update from remote peer 658cb74a815b4006ba57a7c8344125a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:42:21.403891 25615 raft_consensus.cc:1261] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 1 FOLLOWER]: Refusing update from remote peer 658cb74a815b4006ba57a7c8344125a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:42:21.404268 25690 consensus_queue.cc:1035] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed8f3ac405248548ba216bfd2e08bd4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:42:21.404418 25686 consensus_queue.cc:1035] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f19a8013f394e7dbd7d4cd42c14b903" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:42:21.406819 25695 mvcc.cc:205] Tried to move back new op lower bound from 7315450025587310592 to 7315450025490444288. Current Snapshot: MvccSnapshot[applied={T|T < 7315450025587310592}]
I20260806 06:42:21.408895 25183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:42:21.409013 25183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:42:21.409749 25183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 473fb4ff1c694f96aa2ce48a2a28fd08: Sending DeleteTablet for 3 replicas of tablet e6964580d5f9454984be5b270cdd29fa
I20260806 06:42:21.410070 25456 tablet_service.cc:1514] Processing DeleteTablet for tablet e6964580d5f9454984be5b270cdd29fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:42:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:57972
I20260806 06:42:21.410135 25595 tablet_service.cc:1514] Processing DeleteTablet for tablet e6964580d5f9454984be5b270cdd29fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:42:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:47422
I20260806 06:42:21.410140 25316 tablet_service.cc:1514] Processing DeleteTablet for tablet e6964580d5f9454984be5b270cdd29fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:42:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:57494
I20260806 06:42:21.410408 25705 tablet_replica.cc:330] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4: stopping tablet replica
I20260806 06:42:21.410493 25705 raft_consensus.cc:2229] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:42:21.410576 25705 raft_consensus.cc:2258] T e6964580d5f9454984be5b270cdd29fa P fed8f3ac405248548ba216bfd2e08bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:42:21.410591 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 25252
I20260806 06:42:21.411895 25704 tablet_replica.cc:330] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: stopping tablet replica
I20260806 06:42:21.411981 25704 raft_consensus.cc:2229] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:42:21.412083 25704 raft_consensus.cc:2258] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:42:21.412500 25704 ts_tablet_manager.cc:1905] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:42:21.414511 25704 ts_tablet_manager.cc:1918] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:42:21.414587 25704 log.cc:1196] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/e6964580d5f9454984be5b270cdd29fa
I20260806 06:42:21.414871 25704 ts_tablet_manager.cc:1939] T e6964580d5f9454984be5b270cdd29fa P 658cb74a815b4006ba57a7c8344125a4: Deleting consensus metadata
I20260806 06:42:21.415059 25706 tablet_replica.cc:330] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: stopping tablet replica
I20260806 06:42:21.415387 25706 raft_consensus.cc:2229] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:42:21.415443 25171 catalog_manager.cc:4841] TS 658cb74a815b4006ba57a7c8344125a4 (127.14.51.66:37851): tablet e6964580d5f9454984be5b270cdd29fa (table test-table [id=c1acb1d111c1473793478ad8f7c8db84]) successfully deleted
I20260806 06:42:21.415510 25706 raft_consensus.cc:2258] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:42:21.415930 25706 ts_tablet_manager.cc:1905] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:42:21.417800 25706 ts_tablet_manager.cc:1918] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:42:21.417876 25706 log.cc:1196] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/e6964580d5f9454984be5b270cdd29fa
I20260806 06:42:21.418151 25706 ts_tablet_manager.cc:1939] T e6964580d5f9454984be5b270cdd29fa P 3f19a8013f394e7dbd7d4cd42c14b903: Deleting consensus metadata
W20260806 06:42:21.418358 25172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:34913 (error 108)
I20260806 06:42:21.418602 25172 catalog_manager.cc:4841] TS 3f19a8013f394e7dbd7d4cd42c14b903 (127.14.51.67:43571): tablet e6964580d5f9454984be5b270cdd29fa (table test-table [id=c1acb1d111c1473793478ad8f7c8db84]) successfully deleted
W20260806 06:42:21.418917 25676 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:42:21.418770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34913 (local address 127.0.0.1:57508)
0806 06:42:21.418818 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:42:21.418876 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:34913: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:42:21.419054 25172 catalog_manager.cc:4584] TS fed8f3ac405248548ba216bfd2e08bd4 (127.14.51.65:34913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6964580d5f9454984be5b270cdd29fa: Network error: Client connection negotiation failed: client connection to 127.14.51.65:34913: connect: Connection reset by peer (error 104)
I20260806 06:42:21.419260 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 25391
I20260806 06:42:21.426739 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 25530
I20260806 06:42:21.431793 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 25153
2026-08-06T06:42:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2045 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25710](info): set up 2 sockets
Aug 06 06:42:21 dist-test-slave-1xrh krb5kdc[25710](info): commencing operation
krb5kdc: starting...
W20260806 06:42:22.475287 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:42:22 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:42:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangeradmin_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangeradmin_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangerlookup_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangerlookup_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
I20260806 06:42:22.625550 14541 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskN7WBht/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/postgres -l logfile start
2026-08-06 06:42:25.109 UTC [25741] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:42:25.109 UTC [25741] LOG: listening on IPv4 address "127.14.51.84", port 55315
2026-08-06 06:42:25.112 UTC [25741] LOG: listening on Unix socket "/tmp/.s.PGSQL.55315"
2026-08-06 06:42:25.135 UTC [25744] LOG: database system was shut down at 2026-08-06 06:42:23 UTC
2026-08-06 06:42:25.139 UTC [25741] LOG: database system is ready to accept connections
I20260806 06:42:26.159312 14541 mini_postgres.cc:96] Postgres bound to 55315
2026-08-06 06:42:26.167 UTC [25753] FATAL: database "slave" does not exist
127.14.51.84:55315 - accepting connections
I20260806 06:42:26.168020 14541 mini_ranger.cc:161] Starting Ranger...
I20260806 06:42:26.183770 14541 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:42:26.659407 14541 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:42:26.659500 14541 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin
2026-08-06 06:42:26,975 [I] DB FLAVOR :POSTGRES
2026-08-06 06:42:26,975 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:42:26,975 [I] Checking connection..
2026-08-06 06:42:26,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:42:27,226 [I] Checking connection passed.
2026-08-06 06:42:27,226 [I] --------- Verifying version history table ---------
2026-08-06 06:42:27,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:42:27,477 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:42:27,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:42:27,701 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:42:27,701 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:42:27,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:42:27.950 UTC [25818] WARNING: there is no transaction in progress
2026-08-06 06:42:27,957 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:42:27,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:42:28,205 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:42:28,206 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:42:28,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:42:28,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:42:28,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 06:42:28,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:42:29,215 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:42:29,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:42:29,449 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:42:29,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:42:29,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:42:29,912 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:42:29,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:42:30.825 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.842 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.858 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.874 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.890 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.906 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.958 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.965 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.979 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.988 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:30.998 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31.011 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31.020 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31.027 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31.033 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31.040 UTC [25928] WARNING: there is no transaction in progress
2026-08-06 06:42:31,682 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:42:31,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:42:31,934 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:42:31,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:42:32,191 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:42:32,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:42:32,415 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:42:32,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:42:32,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:42:32,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 06:42:33,138 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:42:33,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:42:33,373 [I] DB_PATCHES have already been applied
I20260806 06:42:33.379619 14541 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskN7WBht/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:42:33.379719 14541 mini_ranger.cc:193] Using host: 127.14.51.84
I20260806 06:42:33.383096 14541 mini_ranger.cc:227] Ranger admin URL: http://127.14.51.84:49219
Aug 06, 2026 6:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:42:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:42:34.506877 14541 mini_ranger.cc:160] Time spent starting Ranger: real 8.339s user 0.000s sys 0.010s
Aug 06, 2026 6:42:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:42:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:42:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:42:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:42:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:42:56.670918 14541 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangerkms_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/rangerkms_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.51.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.51.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:42:56 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998576, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:42:56.770381 14541 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:42:56.780867 14541 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:42:57.664438 14541 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:42:57.664527 14541 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms
I20260806 06:42:57.664559 14541 mini_ranger_kms.cc:227] Using postgres at 127.14.51.84:55315
2026-08-06 06:42:58,074 [I] DB FLAVOR :POSTGRES
2026-08-06 06:42:58,074 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:42:58,074 [I] Checking connection
2026-08-06 06:42:58,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:42:58,351 [I] connection success
2026-08-06 06:42:58,351 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:42:58,351 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:42:58,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:42:58,603 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:42:58,603 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:42:58,603 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:42:58,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:42:58,900 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:42:59.355788 14541 mini_ranger_kms.cc:310] Created kms service
I20260806 06:42:59.525095 14541 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:42:59.570158 14541 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:42:59 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998576, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:42:59.980806 14541 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:42:59.980921 14541 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms:/tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:42:59.980942 14541 mini_ranger_kms.cc:240] Using host: 127.14.51.84
I20260806 06:42:59.984431 14541 mini_ranger_kms.cc:276] Ranger KMS PID: 26112
I20260806 06:42:59.984548 14541 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.51.84:40123
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:43:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:43:02.338284 14541 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.568s user 0.000s sys 0.384s
I20260806 06:43:02.338392 14541 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:43:02.338474 14541 mini_ranger_kms.cc:401] 127.14.51.84:40123/kms/v1/keys
Aug 06 06:43:02 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998582, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:43:02 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998582, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
Aug 06 06:43:03 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998582, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
Aug 06 06:43:05 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:43:05.682634 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:39175
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:39175
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:40123/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:05.776116 26165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:05.776317 26165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:05.776357 26165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:05.777644 26165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:05.777690 26165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:05.777705 26165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:05.777719 26165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:05.777731 26165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:05.779531 26165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.51.84:40123/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:39175
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:39175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:05.779754 26165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:05.780017 26165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:43:05.785914 26165 server_base.cc:1034] running on GCE node
W20260806 06:43:05.785841 26170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:05.785774 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:05.785774 26173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:05.786290 26165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:05.786532 26165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:05.787678 26165 hybrid_clock.cc:648] HybridClock initialized: now 1785998585787652 us; error 29 us; skew 500 ppm
Aug 06 06:43:05 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998585, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:05.796643 26165 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:43:05.797123 26165 webserver.cc:460] Webserver started at http://127.14.51.126:42931/ using document root <none> and password file <none>
I20260806 06:43:05.797425 26165 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:05.797585 26165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:05.797757 26165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:05 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998585, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:05.844087 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "ee365d8c69bb4b82abfe5e3bc848029e"
format_stamp: "Formatted at 2026-08-06 06:43:05 on dist-test-slave-1xrh"
server_key: "31adbe14bda49d1a4fa4207efd978e9b"
server_key_iv: "ec5299c25e889e66821dee3c4329dbc3"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:05.844533 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "ee365d8c69bb4b82abfe5e3bc848029e"
format_stamp: "Formatted at 2026-08-06 06:43:05 on dist-test-slave-1xrh"
server_key: "31adbe14bda49d1a4fa4207efd978e9b"
server_key_iv: "ec5299c25e889e66821dee3c4329dbc3"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:05.846133 26165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:43:05.857635 26180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:05.857856 26165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:43:05.857959 26165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "ee365d8c69bb4b82abfe5e3bc848029e"
format_stamp: "Formatted at 2026-08-06 06:43:05 on dist-test-slave-1xrh"
server_key: "31adbe14bda49d1a4fa4207efd978e9b"
server_key_iv: "ec5299c25e889e66821dee3c4329dbc3"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:05.858139 26165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:05.878479 26165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:05.880383 26165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:05.880542 26165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:05.886895 26165 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:39175
I20260806 06:43:05.886935 26234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:39175 every 8 connection(s)
I20260806 06:43:05.887363 26165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:43:05.888185 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:05.888595 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 26165
I20260806 06:43:05.888718 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:43:05.888988 14541 external_mini_cluster.cc:1428] Setting key 1b87943e978eb730658e0a54d7bda4b1
I20260806 06:43:05.892518 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: Bootstrap starting.
I20260806 06:43:05.893579 26235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:05.893971 26235 log.cc:824] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:05.895418 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: No bootstrap required, opened a new log
Aug 06 06:43:05 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:05.898466 26235 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } }
I20260806 06:43:05.898636 26235 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:05.898700 26235 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee365d8c69bb4b82abfe5e3bc848029e, State: Initialized, Role: FOLLOWER
I20260806 06:43:05.898875 26235 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } }
I20260806 06:43:05.898995 26235 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:43:05.899042 26235 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:43:05.899101 26235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:05.899909 26235 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } }
I20260806 06:43:05.900025 26235 leader_election.cc:302] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee365d8c69bb4b82abfe5e3bc848029e; no voters:
I20260806 06:43:05.900226 26235 leader_election.cc:288] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:43:05.900286 26240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:43:05.900554 26235 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:43:05.900900 26240 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [term 1 LEADER]: Becoming Leader. State: Replica: ee365d8c69bb4b82abfe5e3bc848029e, State: Running, Role: LEADER
I20260806 06:43:05.901041 26240 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } }
I20260806 06:43:05.902756 26235 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:43:05.901759 26240 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } } }
I20260806 06:43:05.903515 26240 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:05.903664 26240 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee365d8c69bb4b82abfe5e3bc848029e. Latest consensus state: current_term: 1 leader_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee365d8c69bb4b82abfe5e3bc848029e" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39175 } } }
I20260806 06:43:05.903719 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:05.889587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.14.51.126:39175)
0806 06:43:05.889890 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:05.889899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:05.889933 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:43:05.890508 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:05.890518 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:05.890581 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:05.890734 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:05.891510 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:05.893289 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:05.894434 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:05.894947 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:05.895039 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:05.898209 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:05.898213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:05.898214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:05.898225 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:05.900502 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:05.900897 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:05.900900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:05.900901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:05.901028 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:05.901245 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:05.901245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:05.901246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:05.901762 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:05.901925 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:05.902094 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:43:05.903548 (+ 1454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":130,"threads_started":1}
I20260806 06:43:05.903741 26255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:43:05.903723 26240 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:05.908679 26255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:43:05.911507 26255 catalog_manager.cc:1347] Generated new cluster ID: b98c14cac55f413c9f8f6d0392dce461
I20260806 06:43:05.911576 26255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:43:05.926146 26255 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:43:05.926916 26255 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:43:05.936616 26255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: Generated new TSK 0
I20260806 06:43:05.936861 26255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:43:06.630506 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:40123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:06.729981 26280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:06.730322 26280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:06.730409 26280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:06.731709 26280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:06.731774 26280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:06.731819 26280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:43:06.733394 26280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.51.84:40123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:06.733640 26280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:06.733865 26280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:06.739075 26286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:06.739145 26288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:06.739149 26285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:06.739631 26280 server_base.cc:1034] running on GCE node
I20260806 06:43:06.739825 26280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:06.740046 26280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:06.741192 26280 hybrid_clock.cc:648] HybridClock initialized: now 1785998586741179 us; error 30 us; skew 500 ppm
Aug 06 06:43:06 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998586, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:06.743889 26280 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:43:06.744280 26280 webserver.cc:460] Webserver started at http://127.14.51.65:34961/ using document root <none> and password file <none>
I20260806 06:43:06.744496 26280 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:06.744560 26280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:06.744724 26280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:06 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998586, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:06.759614 26280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "e1fafeaea2234fc38d81b4710f622282"
format_stamp: "Formatted at 2026-08-06 06:43:06 on dist-test-slave-1xrh"
server_key: "48e0200c179ff72a3394be27a7a3e9f9"
server_key_iv: "3b1e23f5316cc1562ad8f2c6e546bfab"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:06.760136 26280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "e1fafeaea2234fc38d81b4710f622282"
format_stamp: "Formatted at 2026-08-06 06:43:06 on dist-test-slave-1xrh"
server_key: "48e0200c179ff72a3394be27a7a3e9f9"
server_key_iv: "3b1e23f5316cc1562ad8f2c6e546bfab"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:06.762467 26280 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 06:43:06.787017 26296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:06.787266 26280 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260806 06:43:06.787369 26280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "e1fafeaea2234fc38d81b4710f622282"
format_stamp: "Formatted at 2026-08-06 06:43:06 on dist-test-slave-1xrh"
server_key: "48e0200c179ff72a3394be27a7a3e9f9"
server_key_iv: "3b1e23f5316cc1562ad8f2c6e546bfab"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:06.787463 26280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:06.812585 26280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:06.814947 26280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:06.815110 26280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:06.815378 26280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:06.815701 26280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:06.815764 26280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:06.815807 26280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:06.815841 26280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:06.826680 26280 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:42131
I20260806 06:43:06.827277 26280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:43:06.827913 26410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:42131 every 8 connection(s)
I20260806 06:43:06.832140 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 26280
I20260806 06:43:06.832252 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:43:06.832456 14541 external_mini_cluster.cc:1428] Setting key 62ca0a263db5dd0019be940d8d89c3d3
Aug 06 06:43:06 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998586, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:06.841607 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:06.829272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:41011 (local address 127.14.51.126:39175)
0806 06:43:06.829506 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:06.829508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:06.829708 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:43:06.830536 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:06.830537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:06.830553 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:06.830618 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:06.831515 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:06.832345 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:06.834359 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:06.834574 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:06.834659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:06.837396 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:06.837397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:06.837397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:06.837405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:06.839662 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:06.840217 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:06.840218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:06.840219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:06.840297 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:06.840669 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:06.840670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:06.840670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:06.840820 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:06.840906 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:06.841384 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 06:43:06.841479 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260806 06:43:06.841782 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:06.829251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39175 (local address 127.14.51.65:41011)
0806 06:43:06.829576 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:06.829610 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:06.830363 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:06.830741 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:06.830746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:06.831047 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:06.831398 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:06.831404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:06.832473 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:06.832474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:06.834247 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:06.834250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:06.834668 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:06.834669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:06.834841 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:06.835538 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:06.835546 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:06.837272 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:06.839769 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:06.839776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:06.839778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:06.840106 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:06.840419 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:06.840420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:06.840420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:06.840584 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:06.840912 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:06.840928 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:06.841163 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 06:43:06.841300 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 06:43:06.841457 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":99,"threads_started":1}
I20260806 06:43:06.842646 26412 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39175
I20260806 06:43:06.842839 26412 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:06.843138 26412 heartbeater.cc:511] Master 127.14.51.126:39175 requested a full tablet report, sending...
I20260806 06:43:06.843837 26197 ts_manager.cc:194] Registered new tserver with Master: e1fafeaea2234fc38d81b4710f622282 (127.14.51.65:42131)
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
I20260806 06:43:06.844671 26197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:41011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:43:06.863494 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:40123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:06.997331 26418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:06.997574 26418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:06.997618 26418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:06.998860 26418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:06.998929 26418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:06.998976 26418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:43:07.000837 26418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.51.84:40123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:07.001107 26418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:07.001411 26418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:07.007540 26426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:07.007874 26425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:07.007874 26428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:07.008888 26418 server_base.cc:1034] running on GCE node
I20260806 06:43:07.009140 26418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:07.009430 26418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:07.010601 26418 hybrid_clock.cc:648] HybridClock initialized: now 1785998587010585 us; error 32 us; skew 500 ppm
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:07.013721 26418 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:43:07.014163 26418 webserver.cc:460] Webserver started at http://127.14.51.66:41843/ using document root <none> and password file <none>
I20260806 06:43:07.014448 26418 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:07.014529 26418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:07.014680 26418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:07.027678 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "b1610bcd9f374e758366b07885eef968"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "9824cc9ed974f9a6847eb36b09c2dc16"
server_key_iv: "7842b7d630a1f64074a7ad97f12007de"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.028977 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "b1610bcd9f374e758366b07885eef968"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "9824cc9ed974f9a6847eb36b09c2dc16"
server_key_iv: "7842b7d630a1f64074a7ad97f12007de"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.032364 26418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:07.051026 26436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.051332 26418 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 06:43:07.051427 26418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "b1610bcd9f374e758366b07885eef968"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "9824cc9ed974f9a6847eb36b09c2dc16"
server_key_iv: "7842b7d630a1f64074a7ad97f12007de"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.051517 26418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:07.104367 26418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:07.106925 26418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:07.107177 26418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:07.107555 26418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:07.108084 26418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:07.108183 26418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.108247 26418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:07.108299 26418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.118919 26418 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:35495
I20260806 06:43:07.119031 26550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:35495 every 8 connection(s)
I20260806 06:43:07.119498 26418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:43:07.122184 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 26418
I20260806 06:43:07.122303 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:43:07.122522 14541 external_mini_cluster.cc:1428] Setting key b20ee6b4f35ed38cae54994123e8f63c
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:07.131870 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.120363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:50869 (local address 127.14.51.126:39175)
0806 06:43:07.120413 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.120415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.120786 (+ 371us) server_negotiation.cc:407] Connection header received
0806 06:43:07.121585 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.121586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.121603 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.121674 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:07.122446 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.123359 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.125083 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.125446 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.125488 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.128434 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:07.128435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:07.128436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:07.128443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:07.130200 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.130717 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.130719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.130719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.130780 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.131059 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.131060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.131061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.131213 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:07.131259 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.131639 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.131737 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:43:07.132658 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.120351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39175 (local address 127.14.51.66:50869)
0806 06:43:07.120610 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.120660 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.121460 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.121750 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.121755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.122024 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:07.122326 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.122332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.123490 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.123490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.124968 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.124971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.125579 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.125580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.125727 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.126374 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:07.126384 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:07.128307 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:07.130298 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:07.130305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:07.130307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:07.130623 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:07.130855 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:07.130856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:07.130856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:07.130982 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:07.131308 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:07.131322 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:07.131472 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.132216 (+ 744us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.132386 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":152,"threads_started":1}
I20260806 06:43:07.133358 26551 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39175
I20260806 06:43:07.133460 26551 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:07.133630 26551 heartbeater.cc:511] Master 127.14.51.126:39175 requested a full tablet report, sending...
I20260806 06:43:07.134014 26197 ts_manager.cc:194] Registered new tserver with Master: b1610bcd9f374e758366b07885eef968 (127.14.51.66:35495)
I20260806 06:43:07.134482 26197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:50869
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:43:07.166303 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:40123/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:07.301576 26558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:07.301927 26558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:07.302024 26558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:07.303656 26558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:07.303761 26558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:07.303865 26558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:43:07.306149 26558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.51.84:40123/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:07.306672 26558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:07.306949 26558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:07.319316 26564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:07.319717 26563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:07.320976 26558 server_base.cc:1034] running on GCE node
W20260806 06:43:07.324287 26566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:07.324644 26558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:07.324915 26558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:07.335307 26558 hybrid_clock.cc:648] HybridClock initialized: now 1785998587332640 us; error 2676 us; skew 500 ppm
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:07.339027 26558 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:43:07.339566 26558 webserver.cc:460] Webserver started at http://127.14.51.67:41319/ using document root <none> and password file <none>
I20260806 06:43:07.339809 26558 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:07.339856 26558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:07.339979 26558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:07.366243 26558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "760c002092dc45d69439b98023fb05af"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "b56b8c22910193ae82ffa3c0a1903ff1"
server_key_iv: "45e80bf16e3651f158ca016ef57daf23"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.366910 26558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "760c002092dc45d69439b98023fb05af"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "b56b8c22910193ae82ffa3c0a1903ff1"
server_key_iv: "45e80bf16e3651f158ca016ef57daf23"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.375558 26558 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260806 06:43:07.392253 26573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.392485 26558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:43:07.392577 26558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "760c002092dc45d69439b98023fb05af"
format_stamp: "Formatted at 2026-08-06 06:43:07 on dist-test-slave-1xrh"
server_key: "b56b8c22910193ae82ffa3c0a1903ff1"
server_key_iv: "45e80bf16e3651f158ca016ef57daf23"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:07.392666 26558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:07.449437 26558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:07.452488 26558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:07.452728 26558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:07.453166 26558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:07.453635 26558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:07.453720 26558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.453790 26558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:07.453856 26558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:07.467198 26558 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:46789
I20260806 06:43:07.467675 26558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:43:07.472440 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 26558
I20260806 06:43:07.472561 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:43:07.472759 14541 external_mini_cluster.cc:1428] Setting key 9f41a608bb2bb984a8d589ea8bba15db
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:07.479386 26689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:46789 every 8 connection(s)
I20260806 06:43:07.480894 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.468949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46249 (local address 127.14.51.126:39175)
0806 06:43:07.468994 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.468997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.469525 (+ 528us) server_negotiation.cc:407] Connection header received
0806 06:43:07.470308 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.470309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.470322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.470352 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:07.471418 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.472208 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.474580 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.474896 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.474937 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.477681 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:07.477683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:07.477683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:07.477690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:07.479429 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.479893 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.479895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.479895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.479954 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.480210 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.480211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.480212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.480332 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:07.480372 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.480709 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.480787 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:43:07.481601 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.468938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39175 (local address 127.14.51.67:46249)
0806 06:43:07.469236 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.469287 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.470127 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.470528 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.470532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.470883 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:07.471306 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.471312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.472329 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.472330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.474478 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.474481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.475022 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.475023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.475184 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.475821 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:07.475831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:07.477575 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:07.479524 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:07.479528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:07.479530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:07.479791 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:07.480023 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:07.480024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:07.480024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:07.480142 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:07.480415 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:07.480420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:07.480567 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.481224 (+ 657us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.481379 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":207,"threads_started":1}
I20260806 06:43:07.482542 26690 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39175
I20260806 06:43:07.482626 26690 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:07.482803 26690 heartbeater.cc:511] Master 127.14.51.126:39175 requested a full tablet report, sending...
I20260806 06:43:07.483270 26197 ts_manager.cc:194] Registered new tserver with Master: 760c002092dc45d69439b98023fb05af (127.14.51.67:46789)
I20260806 06:43:07.483736 26197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:46249
I20260806 06:43:07.485126 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:43:07.500609 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.487167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.14.51.126:39175)
0806 06:43:07.487213 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.487215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.491339 (+ 4124us) server_negotiation.cc:407] Connection header received
0806 06:43:07.491503 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.491504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.491526 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.491567 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:07.492574 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.493410 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.495116 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.495534 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.495584 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.497296 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:07.497297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:07.497298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:07.497359 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:07.499414 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.499806 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.499807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.499808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.499862 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:07.500053 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:07.500054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:07.500054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:07.500168 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:07.500235 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.500449 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.500520 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:43:07.502724 26197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:43:07.505038 26197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:43:07.514741 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:35495 (local address 127.0.0.1:48842)
0806 06:43:07.509587 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.509597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.509668 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.509989 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.509992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.509994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:07.510051 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.510052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.511843 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.511844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.513361 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.513374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.514479 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.514480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.514555 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.514566 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.514627 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.514671 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"thread_start_us":93,"threads_started":1}
I20260806 06:43:07.515192 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.14.51.66:35495)
0806 06:43:07.509208 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.509213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.509643 (+ 430us) server_negotiation.cc:407] Connection header received
0806 06:43:07.509765 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.509767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.509782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.509830 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:07.510283 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.511745 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.513489 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.514286 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.514336 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.514418 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.515019 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.515058 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":104,"threads_started":1}
I20260806 06:43:07.516156 26485 tablet_service.cc:1467] Processing CreateTablet for tablet 99da0114b40a4252b2f02a5dfcf5acf2 (DEFAULT_TABLE table=test-table [id=e80e1f5be4d24e0cbccb4e17d8200220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:07.516535 26485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da0114b40a4252b2f02a5dfcf5acf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:07.520305 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:46789 (local address 127.0.0.1:46638)
0806 06:43:07.509313 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.509351 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.509419 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.516197 (+ 6778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.516198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.516200 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:07.516237 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.516238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.517673 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.517674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.518722 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.518724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.520110 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.520111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.520169 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.520173 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.520201 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.520236 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"thread_start_us":109,"threads_started":1}
I20260806 06:43:07.521008 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.14.51.67:46789)
0806 06:43:07.516040 (+ 7837us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.516045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.516055 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:43:07.516090 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.516092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.516106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.516309 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:07.516460 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.517573 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.518995 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.520008 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.520787 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.520875 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.520900 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.520936 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7798,"thread_start_us":78,"threads_started":1}
I20260806 06:43:07.521729 26623 tablet_service.cc:1467] Processing CreateTablet for tablet 99da0114b40a4252b2f02a5dfcf5acf2 (DEFAULT_TABLE table=test-table [id=e80e1f5be4d24e0cbccb4e17d8200220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:07.522030 26623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da0114b40a4252b2f02a5dfcf5acf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:07.523227 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42131 (local address 127.0.0.1:48114)
0806 06:43:07.510166 (+ 1595us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.510175 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.510212 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.519455 (+ 9243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.519456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.519459 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:07.519517 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.519519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.520905 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.520906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.522039 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.522041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.522991 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.522992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.523054 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.523058 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.523099 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.523156 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1568,"thread_start_us":54,"threads_started":1}
I20260806 06:43:07.523782 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.508657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.14.51.65:42131)
0806 06:43:07.519260 (+ 10603us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.519264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.519277 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:43:07.519310 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.519312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.519327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.519602 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:07.519694 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.520801 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.522150 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.522885 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.523580 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.523645 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.523670 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.523710 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10553,"thread_start_us":57,"threads_started":1}
I20260806 06:43:07.524363 26344 tablet_service.cc:1467] Processing CreateTablet for tablet 99da0114b40a4252b2f02a5dfcf5acf2 (DEFAULT_TABLE table=test-table [id=e80e1f5be4d24e0cbccb4e17d8200220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:07.524693 26344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da0114b40a4252b2f02a5dfcf5acf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:07.526351 26709 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Bootstrap starting.
I20260806 06:43:07.527654 26709 tablet_bootstrap.cc:654] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:07.528241 26709 log.cc:824] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:07.529098 26709 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: No bootstrap required, opened a new log
I20260806 06:43:07.529158 26709 ts_tablet_manager.cc:1397] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:43:07.531930 26709 raft_consensus.cc:348] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.532184 26709 raft_consensus.cc:374] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:07.532265 26709 raft_consensus.cc:729] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760c002092dc45d69439b98023fb05af, State: Initialized, Role: FOLLOWER
I20260806 06:43:07.532454 26709 consensus_queue.cc:260] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.532814 26709 ts_tablet_manager.cc:1428] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 06:43:07.532963 26690 heartbeater.cc:503] Master 127.14.51.126:39175 was elected leader, sending a full tablet report...
I20260806 06:43:07.534372 26710 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Bootstrap starting.
I20260806 06:43:07.535251 26710 tablet_bootstrap.cc:654] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:07.535476 26708 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Bootstrap starting.
I20260806 06:43:07.537356 26710 log.cc:824] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:07.537722 26708 tablet_bootstrap.cc:654] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:07.538203 26708 log.cc:824] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:07.539094 26710 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: No bootstrap required, opened a new log
I20260806 06:43:07.539247 26710 ts_tablet_manager.cc:1397] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260806 06:43:07.542243 26710 raft_consensus.cc:348] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.542498 26710 raft_consensus.cc:374] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:07.542588 26710 raft_consensus.cc:729] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fafeaea2234fc38d81b4710f622282, State: Initialized, Role: FOLLOWER
I20260806 06:43:07.542783 26710 consensus_queue.cc:260] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.543200 26710 ts_tablet_manager.cc:1428] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 06:43:07.543422 26412 heartbeater.cc:503] Master 127.14.51.126:39175 was elected leader, sending a full tablet report...
I20260806 06:43:07.544124 26708 tablet_bootstrap.cc:492] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: No bootstrap required, opened a new log
I20260806 06:43:07.544196 26708 ts_tablet_manager.cc:1397] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 06:43:07.546931 26708 raft_consensus.cc:348] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.547086 26708 raft_consensus.cc:374] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:07.547144 26708 raft_consensus.cc:729] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1610bcd9f374e758366b07885eef968, State: Initialized, Role: FOLLOWER
I20260806 06:43:07.547399 26708 consensus_queue.cc:260] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.547727 26708 ts_tablet_manager.cc:1428] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:43:07.547892 26551 heartbeater.cc:503] Master 127.14.51.126:39175 was elected leader, sending a full tablet report...
W20260806 06:43:07.578691 26413 tablet.cc:2369] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:43:07.584730 26712 raft_consensus.cc:482] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:43:07.584864 26712 raft_consensus.cc:504] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.585342 26712 leader_election.cc:288] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1610bcd9f374e758366b07885eef968 (127.14.51.66:35495), e1fafeaea2234fc38d81b4710f622282 (127.14.51.65:42131)
I20260806 06:43:07.590245 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.585744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:35495 (local address 127.14.51.67:50005)
0806 06:43:07.586187 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.586200 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.586250 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.586785 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.586787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.586789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:07.586902 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.586904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.587981 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.587982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.589183 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.589186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.589952 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.589953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.590077 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.590081 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.590132 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.590171 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421}
I20260806 06:43:07.590939 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.586440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:50005 (local address 127.14.51.66:35495)
0806 06:43:07.586607 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.586609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.586618 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:43:07.586646 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.586647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.586658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.586714 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:07.586986 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.587796 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.589294 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.589837 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.589910 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.589918 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.590830 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.590872 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260806 06:43:07.591333 26505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da0114b40a4252b2f02a5dfcf5acf2" candidate_uuid: "760c002092dc45d69439b98023fb05af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1610bcd9f374e758366b07885eef968" is_pre_election: true
I20260806 06:43:07.591493 26505 raft_consensus.cc:2454] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 760c002092dc45d69439b98023fb05af in term 0.
I20260806 06:43:07.592952 26577 leader_election.cc:302] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 760c002092dc45d69439b98023fb05af, b1610bcd9f374e758366b07885eef968; no voters:
I20260806 06:43:07.593158 26712 raft_consensus.cc:2781] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:43:07.593218 26712 raft_consensus.cc:482] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:43:07.593243 26712 raft_consensus.cc:3037] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:07.594451 26712 raft_consensus.cc:504] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.594579 26712 leader_election.cc:288] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [CANDIDATE]: Term 1 election: Requested vote from peers b1610bcd9f374e758366b07885eef968 (127.14.51.66:35495), e1fafeaea2234fc38d81b4710f622282 (127.14.51.65:42131)
I20260806 06:43:07.594904 26505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da0114b40a4252b2f02a5dfcf5acf2" candidate_uuid: "760c002092dc45d69439b98023fb05af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1610bcd9f374e758366b07885eef968"
I20260806 06:43:07.594998 26505 raft_consensus.cc:3037] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:07.596421 26505 raft_consensus.cc:2454] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 760c002092dc45d69439b98023fb05af in term 1.
I20260806 06:43:07.596692 26577 leader_election.cc:302] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 760c002092dc45d69439b98023fb05af, b1610bcd9f374e758366b07885eef968; no voters:
I20260806 06:43:07.596815 26712 raft_consensus.cc:2781] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:43:07.596977 26712 raft_consensus.cc:686] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 1 LEADER]: Becoming Leader. State: Replica: 760c002092dc45d69439b98023fb05af, State: Running, Role: LEADER
I20260806 06:43:07.597062 26712 consensus_queue.cc:237] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } }
I20260806 06:43:07.597925 26197 catalog_manager.cc:5495] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af reported cstate change: term changed from 0 to 1, leader changed from <none> to 760c002092dc45d69439b98023fb05af (127.14.51.67). New cstate: current_term: 1 leader_uuid: "760c002092dc45d69439b98023fb05af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "760c002092dc45d69439b98023fb05af" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 } health_report { overall_health: UNKNOWN } } }
I20260806 06:43:07.603487 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.585927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42131 (local address 127.14.51.67:60831)
0806 06:43:07.590323 (+ 4396us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.590333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:07.590380 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:07.591505 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:07.591507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:07.591509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:07.591542 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.591543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.601026 (+ 9483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.601027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.602225 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:07.602228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.603245 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.603246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:07.603323 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.603327 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:43:07.603371 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:43:07.603409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4386,"thread_start_us":91,"threads_started":1}
I20260806 06:43:07.603871 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.586552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:60831 (local address 127.14.51.65:42131)
0806 06:43:07.586615 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.586617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.590376 (+ 3759us) server_negotiation.cc:407] Connection header received
0806 06:43:07.590460 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.590462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.590471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.590515 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:07.591650 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.592422 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.602364 (+ 9942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.603081 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.603200 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.603210 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.603729 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.603781 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:43:07.604478 26350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da0114b40a4252b2f02a5dfcf5acf2" candidate_uuid: "760c002092dc45d69439b98023fb05af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fafeaea2234fc38d81b4710f622282" is_pre_election: true
I20260806 06:43:07.604650 26350 raft_consensus.cc:2454] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 760c002092dc45d69439b98023fb05af in term 0.
I20260806 06:43:07.604772 26348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da0114b40a4252b2f02a5dfcf5acf2" candidate_uuid: "760c002092dc45d69439b98023fb05af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fafeaea2234fc38d81b4710f622282"
I20260806 06:43:07.604885 26348 raft_consensus.cc:3037] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:07.605933 26348 raft_consensus.cc:2454] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 760c002092dc45d69439b98023fb05af in term 1.
I20260806 06:43:07.610780 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.606664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.14.51.67:46789)
0806 06:43:07.606824 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:07.606826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:07.606834 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:43:07.606872 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:07.606873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:07.606892 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:07.606958 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:43:07.607271 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.608005 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.609357 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:07.609740 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:07.609820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:07.610224 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:43:07.610370 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:43:07.610474 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:07.610660 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:43:07.610681 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260806 06:43:07.613623 26348 raft_consensus.cc:1261] T 99da0114b40a4252b2f02a5dfcf5acf2 P e1fafeaea2234fc38d81b4710f622282 [term 1 FOLLOWER]: Refusing update from remote peer 760c002092dc45d69439b98023fb05af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:07.613950 26505 raft_consensus.cc:1261] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 1 FOLLOWER]: Refusing update from remote peer 760c002092dc45d69439b98023fb05af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:07.614368 26712 consensus_queue.cc:1035] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1610bcd9f374e758366b07885eef968" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 35495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:07.614531 26712 consensus_queue.cc:1035] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fafeaea2234fc38d81b4710f622282" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:07.620644 26721 mvcc.cc:205] Tried to move back new op lower bound from 7315450214851801088 to 7315450214787379200. Current Snapshot: MvccSnapshot[applied={T|T < 7315450214851801088}]
W20260806 06:43:07.621001 26553 tablet.cc:2369] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:43:07.624205 26196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:43:07.624305 26196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:43:07.625037 26196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee365d8c69bb4b82abfe5e3bc848029e: Sending DeleteTablet for 3 replicas of tablet 99da0114b40a4252b2f02a5dfcf5acf2
I20260806 06:43:07.625725 26485 tablet_service.cc:1514] Processing DeleteTablet for tablet 99da0114b40a4252b2f02a5dfcf5acf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:43:07 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48842
I20260806 06:43:07.626451 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 26280
I20260806 06:43:07.631992 26623 tablet_service.cc:1514] Processing DeleteTablet for tablet 99da0114b40a4252b2f02a5dfcf5acf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:43:07 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46638
I20260806 06:43:07.637478 26725 mvcc.cc:205] Tried to move back new op lower bound from 7315450214851801088 to 7315450214787379200. Current Snapshot: MvccSnapshot[applied={T|T < 7315450214851801088}]
I20260806 06:43:07.639209 26732 tablet_replica.cc:330] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: stopping tablet replica
I20260806 06:43:07.639331 26732 raft_consensus.cc:2229] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:43:07.639453 26732 raft_consensus.cc:2258] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:43:07.639966 26732 ts_tablet_manager.cc:1905] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:43:07.641467 26731 tablet_replica.cc:330] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: stopping tablet replica
I20260806 06:43:07.641562 26731 raft_consensus.cc:2229] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 06 06:43:07 dist-test-slave-1xrh krb5kdc[25710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:07.641667 26731 raft_consensus.cc:2258] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:43:07.643450 26731 ts_tablet_manager.cc:1905] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:43:07.645081 26731 ts_tablet_manager.cc:1918] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:43:07.645160 26731 log.cc:1196] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/99da0114b40a4252b2f02a5dfcf5acf2
I20260806 06:43:07.645414 26731 ts_tablet_manager.cc:1939] T 99da0114b40a4252b2f02a5dfcf5acf2 P b1610bcd9f374e758366b07885eef968: Deleting consensus metadata
W20260806 06:43:07.645712 26184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:42131 (error 108)
I20260806 06:43:07.646472 26184 catalog_manager.cc:4841] TS b1610bcd9f374e758366b07885eef968 (127.14.51.66:35495): tablet 99da0114b40a4252b2f02a5dfcf5acf2 (table test-table [id=e80e1f5be4d24e0cbccb4e17d8200220]) successfully deleted
I20260806 06:43:07.646642 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 26418
I20260806 06:43:07.651536 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:07.651392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42131 (local address 127.0.0.1:48128)
0806 06:43:07.651445 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:07.651499 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:43:07.651666 26184 catalog_manager.cc:4584] TS e1fafeaea2234fc38d81b4710f622282 (127.14.51.65:42131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99da0114b40a4252b2f02a5dfcf5acf2: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42131: connect: Connection refused (error 111)
I20260806 06:43:07.658375 26732 ts_tablet_manager.cc:1918] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:43:07.658481 26732 log.cc:1196] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/99da0114b40a4252b2f02a5dfcf5acf2
I20260806 06:43:07.658798 26732 ts_tablet_manager.cc:1939] T 99da0114b40a4252b2f02a5dfcf5acf2 P 760c002092dc45d69439b98023fb05af: Deleting consensus metadata
I20260806 06:43:07.659260 26183 catalog_manager.cc:4841] TS 760c002092dc45d69439b98023fb05af (127.14.51.67:46789): tablet 99da0114b40a4252b2f02a5dfcf5acf2 (table test-table [id=e80e1f5be4d24e0cbccb4e17d8200220]) successfully deleted
I20260806 06:43:07.661235 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 26558
I20260806 06:43:07.674147 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 26165
I20260806 06:43:07.683283 14541 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:43:08.027410 14541 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:43:08.027527 14541 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:43:08.371498 14541 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:43:08.371 UTC [25741] LOG: received smart shutdown request
2026-08-06 06:43:08.375 UTC [25741] LOG: background worker "logical replication launcher" (PID 25751) exited with exit code 1
2026-08-06 06:43:08.375 UTC [25745] LOG: shutting down
2026-08-06 06:43:08.535 UTC [25741] LOG: database system is shut down
2026-08-06T06:43:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47214 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:43:08 dist-test-slave-1xrh krb5kdc[26743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:43:08 dist-test-slave-1xrh krb5kdc[26743](info): set up 2 sockets
Aug 06 06:43:08 dist-test-slave-1xrh krb5kdc[26743](info): commencing operation
krb5kdc: starting...
W20260806 06:43:09.726625 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:43:09 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:43:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangeradmin_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangeradmin_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangerlookup_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangerlookup_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
I20260806 06:43:09.889928 14541 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskN7WBht/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/postgres -l logfile start
2026-08-06 06:43:12.305 UTC [26773] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:43:12.305 UTC [26773] LOG: listening on IPv4 address "127.14.51.84", port 55893
2026-08-06 06:43:12.309 UTC [26773] LOG: listening on Unix socket "/tmp/.s.PGSQL.55893"
2026-08-06 06:43:12.329 UTC [26776] LOG: database system was shut down at 2026-08-06 06:43:10 UTC
2026-08-06 06:43:12.333 UTC [26773] LOG: database system is ready to accept connections
I20260806 06:43:13.295955 14541 mini_postgres.cc:96] Postgres bound to 55893
2026-08-06 06:43:13.302 UTC [26784] FATAL: database "slave" does not exist
127.14.51.84:55893 - accepting connections
I20260806 06:43:13.302557 14541 mini_ranger.cc:161] Starting Ranger...
I20260806 06:43:13.317487 14541 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:43:13.775218 14541 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:43:13.775302 14541 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin
2026-08-06 06:43:14,042 [I] DB FLAVOR :POSTGRES
2026-08-06 06:43:14,043 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:43:14,043 [I] Checking connection..
2026-08-06 06:43:14,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:43:14,300 [I] Checking connection passed.
2026-08-06 06:43:14,300 [I] --------- Verifying version history table ---------
2026-08-06 06:43:14,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:43:14,559 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:43:14,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:43:14,814 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:43:14,814 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:43:14,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:43:15.064 UTC [26849] WARNING: there is no transaction in progress
2026-08-06 06:43:15,070 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:43:15,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:43:15,307 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:43:15,307 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:43:15,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:43:15,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:43:15,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 06:43:15,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:43:16,241 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:43:16,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:43:16,485 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:43:16,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:43:16,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:43:16,978 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:43:16,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:43:17.914 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:17.931 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:17.947 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:17.961 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:17.977 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:17.996 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.055 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.061 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.075 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.084 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.094 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.110 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.120 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.126 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.132 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18.139 UTC [26962] WARNING: there is no transaction in progress
2026-08-06 06:43:18,753 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:43:18,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:43:18,969 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:43:18,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:43:19,199 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:43:19,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:43:19,462 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:43:19,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:43:19,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:43:19,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 06:43:20,204 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:43:20,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:43:20,443 [I] DB_PATCHES have already been applied
I20260806 06:43:20.449410 14541 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskN7WBht/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:43:20.449468 14541 mini_ranger.cc:193] Using host: 127.14.51.84
I20260806 06:43:20.452699 14541 mini_ranger.cc:227] Ranger admin URL: http://127.14.51.84:60871
Aug 06, 2026 6:43:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:43:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:43:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:43:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:43:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:43:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:43:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:43:21.572935 14541 mini_ranger.cc:160] Time spent starting Ranger: real 8.270s user 0.000s sys 0.008s
Aug 06, 2026 6:43:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:43:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:43:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:43:41 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:43:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:43:42.966715 14541 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.51.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.51.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangerkms_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.51.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/rangerkms_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.51.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.51.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.51.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/HTTP_127.14.51.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:43:43 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998623, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:43:43.047632 14541 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:43:43.056259 14541 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:43:43.920514 14541 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:43:43.920603 14541 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms
I20260806 06:43:43.920624 14541 mini_ranger_kms.cc:227] Using postgres at 127.14.51.84:55893
2026-08-06 06:43:44,088 [I] DB FLAVOR :POSTGRES
2026-08-06 06:43:44,088 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:43:44,088 [I] Checking connection
2026-08-06 06:43:44,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:43:44,320 [I] connection success
2026-08-06 06:43:44,320 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:43:44,320 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:43:44,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:43:44,574 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:43:44,574 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:43:44,574 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:43:44,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.51.84:55893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:43:44,823 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:43:45.300908 14541 mini_ranger_kms.cc:310] Created kms service
I20260806 06:43:45.518093 14541 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:43:45.561261 14541 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:43:45 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998623, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:45.937713 14541 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:43:45.937985 14541 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms:/tmp/dist-test-taskN7WBht/build/release/bin/postgresql.jar:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskN7WBht/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:43:45.938010 14541 mini_ranger_kms.cc:240] Using host: 127.14.51.84
I20260806 06:43:45.941614 14541 mini_ranger_kms.cc:276] Ranger KMS PID: 27143
I20260806 06:43:45.941691 14541 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.51.84:35001
Aug 06, 2026 6:43:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:43:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:43:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:43:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:43:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:43:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:43:48.302506 14541 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.255s user 0.000s sys 0.151s
I20260806 06:43:48.302601 14541 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:43:48.302644 14541 mini_ranger_kms.cc:401] 127.14.51.84:35001/kms/v1/keys
Aug 06 06:43:48 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:43:48 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
Aug 06 06:43:49 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.51.84@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
Aug 06 06:43:51 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:43:51.455967 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43713
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43713
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:51.535261 27195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:51.535507 27195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:51.535554 27195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:51.536746 27195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:51.536810 27195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:51.536835 27195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:51.536866 27195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:51.536890 27195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:51.538342 27195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.51.84:35001/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43713
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:51.538604 27195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:51.538829 27195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:51.543624 27200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:51.543617 27203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:51.543690 27201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:51.543754 27195 server_base.cc:1034] running on GCE node
I20260806 06:43:51.544106 27195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:51.544415 27195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:51.545565 27195 hybrid_clock.cc:648] HybridClock initialized: now 1785998631545544 us; error 39 us; skew 500 ppm
Aug 06 06:43:51 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998631, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:51.548050 27195 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:43:51.548480 27195 webserver.cc:460] Webserver started at http://127.14.51.126:45159/ using document root <none> and password file <none>
I20260806 06:43:51.548647 27195 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:51.548681 27195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:51.548766 27195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:51 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998631, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:51.593067 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "06eac8bec3634abd9ce4e75c5132ce71"
format_stamp: "Formatted at 2026-08-06 06:43:51 on dist-test-slave-1xrh"
server_key: "cc8ef529401fa525ac03750f03b0e352"
server_key_iv: "8b678b7da2482549c79b3140438cb227"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:51.593511 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "06eac8bec3634abd9ce4e75c5132ce71"
format_stamp: "Formatted at 2026-08-06 06:43:51 on dist-test-slave-1xrh"
server_key: "cc8ef529401fa525ac03750f03b0e352"
server_key_iv: "8b678b7da2482549c79b3140438cb227"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:51.595090 27195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:43:51.606329 27210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:51.606567 27195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:51.606679 27195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "06eac8bec3634abd9ce4e75c5132ce71"
format_stamp: "Formatted at 2026-08-06 06:43:51 on dist-test-slave-1xrh"
server_key: "cc8ef529401fa525ac03750f03b0e352"
server_key_iv: "8b678b7da2482549c79b3140438cb227"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:51.606760 27195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:51.618374 27195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:51.620304 27195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:51.620457 27195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:51.624676 27195 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:43713
I20260806 06:43:51.624716 27262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:43713 every 8 connection(s)
I20260806 06:43:51.625042 27195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:43:51.625639 27263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:51.628010 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Bootstrap starting.
I20260806 06:43:51.628579 27263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:51.628871 27263 log.cc:824] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:51.629540 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: No bootstrap required, opened a new log
I20260806 06:43:51.631289 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27195
I20260806 06:43:51.631390 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:43:51.631567 14541 external_mini_cluster.cc:1428] Setting key e6a4df036a358f0f86295f25299ac978
I20260806 06:43:51.631682 27263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:51.631876 27263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:51.631917 27263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06eac8bec3634abd9ce4e75c5132ce71, State: Initialized, Role: FOLLOWER
I20260806 06:43:51.632072 27263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:51.632153 27263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:43:51.632203 27263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:43:51.632256 27263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:51.632738 27263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:51.632884 27263 leader_election.cc:302] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06eac8bec3634abd9ce4e75c5132ce71; no voters:
I20260806 06:43:51.633065 27263 leader_election.cc:288] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:43:51.633109 27268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:43:51.633217 27268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 LEADER]: Becoming Leader. State: Replica: 06eac8bec3634abd9ce4e75c5132ce71, State: Running, Role: LEADER
I20260806 06:43:51.633323 27263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:43:51.633375 27268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:51.633805 27270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06eac8bec3634abd9ce4e75c5132ce71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } } }
I20260806 06:43:51.633896 27270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:51.634136 27269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06eac8bec3634abd9ce4e75c5132ce71. Latest consensus state: current_term: 1 leader_uuid: "06eac8bec3634abd9ce4e75c5132ce71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } } }
I20260806 06:43:51.634239 27269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:51.634403 27276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:43:51.634968 27276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:43:51.635291 27263 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:43:51.639755 27276 catalog_manager.cc:1347] Generated new cluster ID: b353b78ca74a49b2b7c73738a07b275d
I20260806 06:43:51.639815 27276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:43:51 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:51.646271 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:51.632088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.14.51.126:43713)
0806 06:43:51.632272 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:51.632281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:51.632308 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:43:51.632414 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:51.632419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:51.632463 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:51.632589 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:51.633392 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:51.634887 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:51.637076 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:51.638997 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:51.639183 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:51.642065 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:51.642070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:51.642070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:51.642082 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:51.644194 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:51.644647 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:51.644650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:51.644650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:51.644781 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:51.645031 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:51.645032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:51.645033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:51.645420 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:51.645564 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:51.645846 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:43:51.645998 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260806 06:43:51.670056 27276 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:43:51.670758 27276 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:43:51.685439 27276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Generated new TSK 0
I20260806 06:43:51.685618 27276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:43:52.338606 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:52.441241 27309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:52.441421 27309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:52.441443 27309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:52.442502 27309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:52.442539 27309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:52.442562 27309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:43:52.444021 27309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:52.444237 27309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:52.444474 27309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:52.450176 27316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:52.450263 27314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:52.450191 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:52.451131 27309 server_base.cc:1034] running on GCE node
I20260806 06:43:52.451340 27309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:52.451570 27309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:52.452719 27309 hybrid_clock.cc:648] HybridClock initialized: now 1785998632452701 us; error 39 us; skew 500 ppm
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:52.455832 27309 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:43:52.456312 27309 webserver.cc:460] Webserver started at http://127.14.51.65:38491/ using document root <none> and password file <none>
I20260806 06:43:52.456661 27309 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:52.456722 27309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:52.456836 27309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:52.470791 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "fc319634c54f439185fea0ca4062a4f0"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "60e4059d11f797df2eaaecbf886391b5"
server_key_iv: "e120a01c92c051e1849f64ebb096cb55"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.472680 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "fc319634c54f439185fea0ca4062a4f0"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "60e4059d11f797df2eaaecbf886391b5"
server_key_iv: "e120a01c92c051e1849f64ebb096cb55"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.477524 27309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260806 06:43:52.491236 27327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.491567 27309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 06:43:52.491674 27309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "fc319634c54f439185fea0ca4062a4f0"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "60e4059d11f797df2eaaecbf886391b5"
server_key_iv: "e120a01c92c051e1849f64ebb096cb55"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.491770 27309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:52.511067 27309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:52.513576 27309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:52.513754 27309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:52.514027 27309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:52.514446 27309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:52.514534 27309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.514626 27309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:52.514699 27309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.521720 27309 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:33001
I20260806 06:43:52.522161 27309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:43:52.524219 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27309
I20260806 06:43:52.524343 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:43:52.524509 27442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:33001 every 8 connection(s)
I20260806 06:43:52.524559 14541 external_mini_cluster.cc:1428] Setting key 4ace2fb73bddbdf50480c695a249bb9f
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:52.552533 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:52.525434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:38753 (local address 127.14.51.126:43713)
0806 06:43:52.525582 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:52.525584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:52.535411 (+ 9827us) server_negotiation.cc:407] Connection header received
0806 06:43:52.539894 (+ 4483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:52.539896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:52.539912 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:52.539959 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:52.540835 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.541641 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.543162 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.543460 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.543504 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:52.549063 (+ 5559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:52.549064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:52.549064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:52.549071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:52.550916 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:52.551431 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:52.551433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:52.551433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:52.551497 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:52.551775 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:52.551776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:52.551776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:52.551920 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:52.551972 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:52.552324 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 06:43:52.552408 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260806 06:43:52.553316 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:52.525425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.65:38753)
0806 06:43:52.535248 (+ 9823us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:52.535300 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:52.539757 (+ 4457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:52.540149 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:52.540153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:52.540424 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:52.540704 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:52.540711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.541846 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.541847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:52.543037 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:52.543040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.543603 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.543604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:52.543735 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:52.547267 (+ 3532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:52.547286 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:52.548951 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:52.551015 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:52.551022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:52.551024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:52.551326 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:52.551568 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:52.551568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:52.551569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:52.551691 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:52.552007 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:52.552013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:52.552175 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 06:43:52.552901 (+ 726us) client_negotiation.cc:240] Negotiation successful
0806 06:43:52.553082 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9775,"thread_start_us":86,"threads_started":1}
I20260806 06:43:52.554078 27443 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:52.554183 27443 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:52.554360 27443 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:52.554855 27227 ts_manager.cc:194] Registered new tserver with Master: fc319634c54f439185fea0ca4062a4f0 (127.14.51.65:33001)
I20260806 06:43:52.555763 27227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:38753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:43:52.574939 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 06:43:52.681758 27451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:52.681982 27451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:52.682024 27451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:52.683303 27451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:52.683387 27451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:52.683461 27451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:43:52.685559 27451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:52.685897 27451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:52.686192 27451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:43:52.693001 27451 server_base.cc:1034] running on GCE node
W20260806 06:43:52.693090 27460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:52.693536 27456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:52.693586 27458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:52.693895 27451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:52.694144 27451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:52.695307 27451 hybrid_clock.cc:648] HybridClock initialized: now 1785998632695285 us; error 32 us; skew 500 ppm
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:52.698520 27451 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:43:52.698992 27451 webserver.cc:460] Webserver started at http://127.14.51.66:45771/ using document root <none> and password file <none>
I20260806 06:43:52.699270 27451 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:52.699340 27451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:52.699470 27451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:52.711361 27451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "589fa9205898498aa8e156dc244e13ef"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "eeaebbe7d4198074c9c52bbe474ddda8"
server_key_iv: "b63580535f79e178bad58fd8bb8c8b1a"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.711859 27451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "589fa9205898498aa8e156dc244e13ef"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "eeaebbe7d4198074c9c52bbe474ddda8"
server_key_iv: "b63580535f79e178bad58fd8bb8c8b1a"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.713907 27451 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 06:43:52.731271 27467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.731506 27451 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 06:43:52.731606 27451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "589fa9205898498aa8e156dc244e13ef"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "eeaebbe7d4198074c9c52bbe474ddda8"
server_key_iv: "b63580535f79e178bad58fd8bb8c8b1a"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.731680 27451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:52.749274 27451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:52.751756 27451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:52.751947 27451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:52.752234 27451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:52.752641 27451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:52.752736 27451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.752823 27451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:52.752880 27451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.761202 27451 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:45385
I20260806 06:43:52.761602 27451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:43:52.761766 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27451
I20260806 06:43:52.761878 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:43:52.762140 14541 external_mini_cluster.cc:1428] Setting key c48491cdfe33aa5ee3ef01946d67f782
I20260806 06:43:52.767204 27583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:45385 every 8 connection(s)
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:43:52.778923 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:52.767749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:45797 (local address 127.14.51.126:43713)
0806 06:43:52.767798 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:52.767801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:52.768756 (+ 955us) server_negotiation.cc:407] Connection header received
0806 06:43:52.769334 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:52.769335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:52.769351 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:52.769398 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:52.770176 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.771057 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.772265 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.772631 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.772683 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:52.775542 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:52.775544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:52.775544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:52.775553 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:52.777423 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:52.777880 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:52.777881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:52.777882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:52.777941 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:52.778199 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:52.778200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:52.778200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:52.778327 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:52.778406 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:52.778679 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 06:43:52.778769 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:43:52.779161 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:52.767737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.66:45797)
0806 06:43:52.768631 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:52.768666 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:52.769208 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:52.769521 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:52.769525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:52.769779 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:52.770054 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:52.770060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.771179 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.771180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:52.772142 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:52.772145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:52.772795 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:52.772796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:52.772944 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:52.773603 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:52.773614 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:52.775429 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:52.777517 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:52.777524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:52.777526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:52.777744 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:52.778024 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:52.778024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:52.778024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:52.778119 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:52.778396 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:52.778403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:52.778543 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 06:43:52.778660 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:43:52.778853 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
I20260806 06:43:52.779709 27584 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:52.779852 27584 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:52.780095 27584 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:52.780495 27227 ts_manager.cc:194] Registered new tserver with Master: 589fa9205898498aa8e156dc244e13ef (127.14.51.66:45385)
I20260806 06:43:52.781024 27227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:45797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:43:52.797830 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:52.916299 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:52.916663 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:52.916751 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:52.918481 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:52.918713 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:52.918812 27590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:43:52.921065 27590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:52.921514 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:52.921855 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:52.929080 27596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:52.929140 27595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:52.929549 27590 server_base.cc:1034] running on GCE node
W20260806 06:43:52.931253 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:52.931545 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:52.931810 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:52.935451 27590 hybrid_clock.cc:648] HybridClock initialized: now 1785998632935427 us; error 36 us; skew 500 ppm
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:52.939636 27590 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:43:52.940133 27590 webserver.cc:460] Webserver started at http://127.14.51.67:35289/ using document root <none> and password file <none>
I20260806 06:43:52.940371 27590 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:52.940431 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:52.940564 27590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:43:52 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:52.955698 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "fcf9fab26b68e13ac2681dd05d636719"
server_key_iv: "8afe097447e82b996a77afcf9f89f601"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.956300 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "fcf9fab26b68e13ac2681dd05d636719"
server_key_iv: "8afe097447e82b996a77afcf9f89f601"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.958504 27590 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 06:43:52.973627 27605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.973858 27590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:43:52.973950 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "fcf9fab26b68e13ac2681dd05d636719"
server_key_iv: "8afe097447e82b996a77afcf9f89f601"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:52.974028 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:52.992076 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:52.994354 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:52.994526 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:52.994797 27590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:52.995314 27590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:52.995412 27590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:52.995460 27590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:52.995481 27590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.006698 27590 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:44671
I20260806 06:43:53.007076 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:43:53.013641 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27590
I20260806 06:43:53.013767 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:43:53.013953 27719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:44671 every 8 connection(s)
I20260806 06:43:53.014012 14541 external_mini_cluster.cc:1428] Setting key d6d3d0984142cb10e84237fa77494d33
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998632, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:53.023033 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.007970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:34509 (local address 127.14.51.126:43713)
0806 06:43:53.008057 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:53.008060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:53.014459 (+ 6399us) server_negotiation.cc:407] Connection header received
0806 06:43:53.015338 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:53.015340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:53.015351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:53.015406 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:53.016127 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.016638 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.017693 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.017896 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.017980 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.020142 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:53.020144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:53.020144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:53.020153 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:53.021625 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.022045 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.022046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.022046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.022111 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.022368 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.022369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.022369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.022486 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:53.022592 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:53.022856 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 06:43:53.022944 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:43:53.023284 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.007944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.67:34509)
0806 06:43:53.014134 (+ 6190us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:53.014340 (+ 206us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:53.015227 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:53.015478 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:53.015483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:53.015784 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:53.016023 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.016029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.016741 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.016742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.017565 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.017568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.018004 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.018006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.018156 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.018718 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:53.018725 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:53.020048 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:53.021728 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.021734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.021737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.021966 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.022181 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.022181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.022182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.022299 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.022586 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:53.022594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:53.022727 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:43:53.022834 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:43:53.022972 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6098,"thread_start_us":116,"threads_started":1}
I20260806 06:43:53.023933 27721 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:53.024051 27721 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.024236 27721 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:53.024626 27227 ts_manager.cc:194] Registered new tserver with Master: 230e21c9a3ea466e9cf17b6f3f9db3ac (127.14.51.67:44671)
I20260806 06:43:53.025015 27227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:34509
I20260806 06:43:53.025827 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:43:53.026455 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27309
I20260806 06:43:53.032923 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27451
I20260806 06:43:53.043462 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27590
I20260806 06:43:53.052170 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27195
I20260806 06:43:53.059791 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43713
--webserver_interface=127.14.51.126
--webserver_port=45159
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:43713
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:53.141530 27724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:53.141717 27724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:53.141737 27724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:53.142870 27724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:53.142911 27724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:53.142925 27724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:53.142938 27724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:53.142949 27724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:53.144542 27724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:43713
--ranger_config_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:43713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:53.144793 27724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:53.145047 27724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:53.150552 27730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.150569 27729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.150540 27732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:53.150813 27724 server_base.cc:1034] running on GCE node
I20260806 06:43:53.151059 27724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:53.151340 27724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:53.152505 27724 hybrid_clock.cc:648] HybridClock initialized: now 1785998633152477 us; error 44 us; skew 500 ppm
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:53.155141 27724 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:43:53.155643 27724 webserver.cc:460] Webserver started at http://127.14.51.126:45159/ using document root <none> and password file <none>
I20260806 06:43:53.155874 27724 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:53.155939 27724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:53.203475 27724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:53.204259 27740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.204438 27724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:53.204531 27724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "06eac8bec3634abd9ce4e75c5132ce71"
format_stamp: "Formatted at 2026-08-06 06:43:51 on dist-test-slave-1xrh"
server_key: "cc8ef529401fa525ac03750f03b0e352"
server_key_iv: "8b678b7da2482549c79b3140438cb227"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:53.204900 27724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:53.220430 27724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:53.222445 27724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:53.222590 27724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:53.227716 27724 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:43713
I20260806 06:43:53.227802 27793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:43713 every 8 connection(s)
I20260806 06:43:53.228168 27724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:43:53.229820 27794 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:43:53.230641 27794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Bootstrap starting.
I20260806 06:43:53.236084 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27724
I20260806 06:43:53.236449 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:33001
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:43713
--webserver_port=38491
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:43:53.240595 27794 log.cc:824] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:53.242686 27794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:43:53.242992 27794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Bootstrap complete.
I20260806 06:43:53.246315 27794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:53.246624 27794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06eac8bec3634abd9ce4e75c5132ce71, State: Initialized, Role: FOLLOWER
I20260806 06:43:53.246812 27794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:53.246889 27794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:43:53.246961 27794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:43:53.247010 27794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:43:53.248342 27794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:53.248450 27794 leader_election.cc:302] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06eac8bec3634abd9ce4e75c5132ce71; no voters:
I20260806 06:43:53.248641 27794 leader_election.cc:288] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:43:53.248701 27798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:43:53.248893 27798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [term 2 LEADER]: Becoming Leader. State: Replica: 06eac8bec3634abd9ce4e75c5132ce71, State: Running, Role: LEADER
I20260806 06:43:53.248961 27794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:43:53.249027 27798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } }
I20260806 06:43:53.249331 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "06eac8bec3634abd9ce4e75c5132ce71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } } }
I20260806 06:43:53.249446 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:53.249625 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06eac8bec3634abd9ce4e75c5132ce71. Latest consensus state: current_term: 2 leader_uuid: "06eac8bec3634abd9ce4e75c5132ce71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06eac8bec3634abd9ce4e75c5132ce71" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 43713 } } }
I20260806 06:43:53.249719 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:53.249899 27806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:43:53.250372 27806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:43:53.250679 27806 catalog_manager.cc:1259] Loaded cluster ID: b353b78ca74a49b2b7c73738a07b275d
I20260806 06:43:53.250758 27806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:43:53.250783 27794 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:43:53.254496 27806 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:43:53.255187 27806 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Loaded TSK: 0
I20260806 06:43:53.255492 27806 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:43:53.347970 27796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:53.348209 27796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:53.348248 27796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:53.349522 27796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:53.349586 27796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:53.349630 27796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:43:53.351341 27796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:33001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=38491
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:53.351583 27796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:53.351815 27796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:53.358150 27832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.358209 27834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:53.359091 27796 server_base.cc:1034] running on GCE node
W20260806 06:43:53.359278 27831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:53.359541 27796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:53.359789 27796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:53.360945 27796 hybrid_clock.cc:648] HybridClock initialized: now 1785998633360920 us; error 40 us; skew 500 ppm
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:53.364044 27796 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:43:53.364544 27796 webserver.cc:460] Webserver started at http://127.14.51.65:38491/ using document root <none> and password file <none>
I20260806 06:43:53.364813 27796 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:53.364877 27796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:53.387638 27796 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 06:43:53.395725 27842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.395923 27796 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260806 06:43:53.396015 27796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "fc319634c54f439185fea0ca4062a4f0"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "60e4059d11f797df2eaaecbf886391b5"
server_key_iv: "e120a01c92c051e1849f64ebb096cb55"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:53.396481 27796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:53.421406 27796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:53.423768 27796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:53.423938 27796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:53.424165 27796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:53.424543 27796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:53.424587 27796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.424618 27796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:53.424665 27796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.432050 27796 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:33001
I20260806 06:43:53.432132 27956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:33001 every 8 connection(s)
I20260806 06:43:53.432554 27796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:43:53.439340 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27796
I20260806 06:43:53.439666 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:45385
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:43713
--webserver_port=45771
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:53.447484 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.433512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.65:50037)
0806 06:43:53.433710 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:53.433754 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:53.434379 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:53.435867 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:53.435872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:53.436096 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:53.436375 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.436382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.438094 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.438095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.440029 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.440032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.441142 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.441143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.441284 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.441795 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:53.441804 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:53.443518 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:53.445645 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.445650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.445652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.445932 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.446187 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.446188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.446188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.446305 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.446811 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:53.446817 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:53.446970 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 06:43:53.447076 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:43:53.447264 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260806 06:43:53.448240 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.433578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:50037 (local address 127.14.51.126:43713)
0806 06:43:53.435373 (+ 1795us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:53.435383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:53.435421 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:43:53.435567 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:53.435574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:53.435636 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:53.435768 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:53.436504 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.437909 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.440141 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.440960 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.441500 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.443632 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:53.443636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:53.443636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:53.443647 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:53.445556 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.446019 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.446022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.446022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.446119 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.446371 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.446371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.446372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.446704 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:53.446781 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:53.447942 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0806 06:43:53.448056 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1739,"thread_start_us":105,"threads_started":1}
I20260806 06:43:53.449508 27957 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:53.449609 27957 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.449790 27957 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.450704 27957 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43713 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:43:53.450770 27957 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.450834 27957 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:53.451236 27957 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.451314 27957 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.451650 27957 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:43:53.549914 27961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:53.550267 27961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:53.550364 27961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:53.551748 27961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:53.551789 27961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:53.551815 27961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:43:53.553264 27961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:45385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=45771
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:53.553503 27961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:53.553738 27961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:53.559724 27966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.559767 27967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.560206 27969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:53.560810 27961 server_base.cc:1034] running on GCE node
I20260806 06:43:53.561040 27961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:53.561343 27961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:53.562510 27961 hybrid_clock.cc:648] HybridClock initialized: now 1785998633562490 us; error 33 us; skew 500 ppm
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:53.565735 27961 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:43:53.566210 27961 webserver.cc:460] Webserver started at http://127.14.51.66:45771/ using document root <none> and password file <none>
I20260806 06:43:53.566510 27961 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:53.566617 27961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:53.585650 27961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:53.586701 27978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.586860 27961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:53.586948 27961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "589fa9205898498aa8e156dc244e13ef"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "eeaebbe7d4198074c9c52bbe474ddda8"
server_key_iv: "b63580535f79e178bad58fd8bb8c8b1a"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:53.587417 27961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:53.604561 27961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:53.606719 27961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:53.606887 27961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:53.607175 27961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:53.607549 27961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:53.607607 27961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.607645 27961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:53.607674 27961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.615512 27961 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:45385
I20260806 06:43:53.615644 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:45385 every 8 connection(s)
I20260806 06:43:53.616042 27961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:53.626187 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 27961
I20260806 06:43:53.626358 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.616805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:53351 (local address 127.14.51.126:43713)
0806 06:43:53.616878 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:53.616880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:53.617169 (+ 289us) server_negotiation.cc:407] Connection header received
0806 06:43:53.617823 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:53.617824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:53.617836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:53.617879 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:53.618575 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.619056 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.620598 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.620789 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.620841 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.623332 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:53.623334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:53.623334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:53.623342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:53.624851 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.625274 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.625275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.625276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.625344 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.625605 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.625606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.625606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.625727 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:53.625802 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:53.626148 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:43:53.626245 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:43:53.626502 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.51.84:35001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:44671
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:43713
--webserver_port=35289
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:43:53.626941 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.616785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.66:53351)
0806 06:43:53.617009 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:53.617058 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:53.617714 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:53.617969 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:53.617973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:53.618205 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:53.618492 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.618498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.619168 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.619169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.620481 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.620484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.620906 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.620908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.621063 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.621637 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:53.621648 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:53.623240 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:53.624965 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.624971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.624973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.625194 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.625413 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.625414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.625414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.625529 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.625792 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:53.625799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:53.625941 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 06:43:53.626494 (+ 553us) client_negotiation.cc:240] Negotiation successful
0806 06:43:53.626671 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260806 06:43:53.628218 28094 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:53.628337 28094 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.628540 28094 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.629320 28094 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43713 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:43:53.629405 28094 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.629477 28094 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:53.629793 28094 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.629870 28094 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.630208 28094 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:43:53.735721 28097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:53.735986 28097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:53.736022 28097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:53.737660 28097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:53.737718 28097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:53.737757 28097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:43:53.739880 28097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.51.84:35001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=35289
--tserver_master_addrs=127.14.51.126:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:53.740100 28097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:53.740387 28097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:53.748107 28102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.748116 28105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:53.748246 28103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:53.748760 28097 server_base.cc:1034] running on GCE node
I20260806 06:43:53.748958 28097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:53.749191 28097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:53.752508 28097 hybrid_clock.cc:648] HybridClock initialized: now 1785998633752484 us; error 37 us; skew 500 ppm
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:53.756088 28097 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:43:53.756564 28097 webserver.cc:460] Webserver started at http://127.14.51.67:35289/ using document root <none> and password file <none>
I20260806 06:43:53.756844 28097 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:53.756946 28097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:53.772194 28097 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:53.774071 28114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.774273 28097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260806 06:43:53.774420 28097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac"
format_stamp: "Formatted at 2026-08-06 06:43:52 on dist-test-slave-1xrh"
server_key: "fcf9fab26b68e13ac2681dd05d636719"
server_key_iv: "8afe097447e82b996a77afcf9f89f601"
server_key_version: "kuduclusterkey@0"
I20260806 06:43:53.774955 28097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:53.811239 28097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:53.813709 28097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:53.813876 28097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:53.814157 28097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:43:53.814535 28097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:43:53.814589 28097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.814621 28097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:43:53.814687 28097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:53.823182 28097 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:44671
I20260806 06:43:53.823258 28227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:44671 every 8 connection(s)
I20260806 06:43:53.823618 28097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:43:53.826159 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28097
Aug 06 06:43:53 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:53.834779 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.824489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:60251 (local address 127.14.51.126:43713)
0806 06:43:53.824560 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:53.824562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:53.824767 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:43:53.825247 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:53.825248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:53.825261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:53.825340 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:53.825859 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.826615 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.828097 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.828414 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.828497 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.831558 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:53.831559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:53.831559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:53.831567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:53.833209 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.833650 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.833651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.833651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.833725 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.834046 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.834047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.834047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.834173 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:53.834218 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:53.834605 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 06:43:53.834680 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:43:53.835270 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.824445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:43713 (local address 127.14.51.67:60251)
0806 06:43:53.824628 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:53.824671 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:53.825140 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:53.825408 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:53.825411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:53.825576 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:53.825761 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.825766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.826714 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.826715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.827999 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:53.828002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.828533 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.828534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:53.828685 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.829774 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:53.829784 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:53.831465 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:53.833303 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.833309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.833310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.833561 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.833793 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:53.833793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:53.833794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:53.833956 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:53.834269 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:53.834275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:53.834430 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 06:43:53.834906 (+ 476us) client_negotiation.cc:240] Negotiation successful
0806 06:43:53.835044 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260806 06:43:53.835955 28228 heartbeater.cc:347] Connected to a master server at 127.14.51.126:43713
I20260806 06:43:53.836050 28228 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.836236 28228 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.836885 28228 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:43713 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:43:53.836962 28228 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.837018 28228 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:53.837342 28228 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:53.837414 28228 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
W20260806 06:43:53.837750 28228 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:43:53.842166 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:53.826595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.14.51.126:43713)
0806 06:43:53.828147 (+ 1552us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:53.828149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:53.828156 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:43:53.828184 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:53.828185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:53.828194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:53.828271 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:53.829072 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.829609 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.831570 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:53.831841 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:53.831898 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:53.835092 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:53.835093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:53.835094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:53.835101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:53.837013 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.839222 (+ 2209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.839223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.839223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.839274 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:53.839484 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:53.839485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:53.839485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:53.839619 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:53.839781 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:53.842036 (+ 2255us) server_negotiation.cc:299] Negotiation successful
0806 06:43:53.842106 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":64,"threads_started":1}
Aug 06 06:43:54 dist-test-slave-1xrh krb5kdc[26743](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:54.451831 27957 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:54.451973 27957 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:54.452347 27757 ts_manager.cc:194] Registered new tserver with Master: fc319634c54f439185fea0ca4062a4f0 (127.14.51.65:33001)
I20260806 06:43:54.453363 27757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:50037
I20260806 06:43:54.630375 28094 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:54.630558 28094 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:54.630893 27757 ts_manager.cc:194] Registered new tserver with Master: 589fa9205898498aa8e156dc244e13ef (127.14.51.66:45385)
I20260806 06:43:54.631436 27757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:53351
Aug 06 06:43:54 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
I20260806 06:43:54.837929 28228 heartbeater.cc:464] Registering TS with master...
I20260806 06:43:54.838088 28228 heartbeater.cc:511] Master 127.14.51.126:43713 requested a full tablet report, sending...
I20260806 06:43:54.838335 27757 ts_manager.cc:194] Registered new tserver with Master: 230e21c9a3ea466e9cf17b6f3f9db3ac (127.14.51.67:44671)
I20260806 06:43:54.838721 27757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:60251
I20260806 06:43:54.846068 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:43:54.856489 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.847141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.14.51.126:43713)
0806 06:43:54.847412 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:54.847413 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:54.847419 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:43:54.847436 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:54.847436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:54.847445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:54.847519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:54.848117 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.848638 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.849368 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.849602 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.849682 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.853704 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:54.853705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:54.853705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:54.853713 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:54.855276 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:54.855638 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:54.855639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:54.855640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:54.855722 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:54.855934 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:54.855935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:54.855936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:54.856052 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:54.856159 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:54.856270 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:43:54.856360 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":71,"threads_started":1}
I20260806 06:43:54.858613 27757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:43:54.860518 27757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:43:54.875804 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:33001 (local address 127.0.0.1:60108)
0806 06:43:54.871220 (+ 7937us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:54.871229 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:54.871304 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:54.871657 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:54.871660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:54.871663 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:54.871730 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.871732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.872991 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.872992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.874316 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.874319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.875488 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.875489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.875586 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.875597 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:43:54.875660 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:43:54.875724 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7921,"thread_start_us":68,"threads_started":1}
I20260806 06:43:54.876062 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.14.51.65:33001)
0806 06:43:54.871238 (+ 7713us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:54.871242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:54.871303 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:43:54.871427 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:54.871430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:54.871456 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:54.871534 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:54.871815 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.872899 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.874423 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.875329 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.875400 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.875456 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:54.875921 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 06:43:54.875969 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7675,"thread_start_us":74,"threads_started":1}
I20260806 06:43:54.876849 27873 tablet_service.cc:1467] Processing CreateTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 (DEFAULT_TABLE table=test-table [id=98e2f1271c294dc8a6b2d665024ef516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:54.877336 27873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cc474d50544d8f965c4f5a0cc578e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:54.877939 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:44671 (local address 127.0.0.1:43988)
0806 06:43:54.867214 (+ 3965us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:54.867257 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:54.867331 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:54.871936 (+ 4605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:54.871937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:54.871938 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:54.871970 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.871972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.873205 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.873206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.876438 (+ 3232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.876441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.877746 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.877747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.877808 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.877811 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:43:54.877841 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:43:54.877876 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3932,"thread_start_us":102,"threads_started":1}
I20260806 06:43:54.878307 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.14.51.67:44671)
0806 06:43:54.871492 (+ 7968us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:54.871497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:54.871507 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:43:54.871821 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:54.871824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:54.871841 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:54.872035 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:54.872175 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.873118 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.876530 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.877633 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.878109 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.878170 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:54.878195 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:43:54.878232 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7928,"thread_start_us":69,"threads_started":1}
I20260806 06:43:54.879024 28162 tablet_service.cc:1467] Processing CreateTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 (DEFAULT_TABLE table=test-table [id=98e2f1271c294dc8a6b2d665024ef516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:54.879385 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45385 (local address 127.0.0.1:32906)
0806 06:43:54.871306 (+ 7546us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:54.871327 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:54.871394 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:54.871705 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:54.871706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:54.871708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:54.871759 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.871761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.873054 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.873055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.877590 (+ 4535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:54.877593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.878642 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.878643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:54.878711 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.878715 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:43:54.879013 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 06:43:54.879323 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7528,"mutex_wait_us":3303,"thread_start_us":105,"threads_started":1}
I20260806 06:43:54.879403 28162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cc474d50544d8f965c4f5a0cc578e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:54.879705 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:54.863949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.14.51.66:45385)
0806 06:43:54.864079 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:54.864084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:54.871463 (+ 7379us) server_negotiation.cc:407] Connection header received
0806 06:43:54.871507 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:54.871510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:54.871528 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:54.871607 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:54.871880 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.872951 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.877696 (+ 4745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:54.878520 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:54.878611 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:54.878663 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:54.879552 (+ 889us) server_negotiation.cc:299] Negotiation successful
0806 06:43:54.879593 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260806 06:43:54.880398 28027 tablet_service.cc:1467] Processing CreateTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 (DEFAULT_TABLE table=test-table [id=98e2f1271c294dc8a6b2d665024ef516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:43:54.880704 28027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cc474d50544d8f965c4f5a0cc578e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:54.881562 28261 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Bootstrap starting.
I20260806 06:43:54.882324 28261 tablet_bootstrap.cc:654] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:54.882608 28261 log.cc:824] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:54.884119 28263 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Bootstrap starting.
I20260806 06:43:54.884855 28264 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Bootstrap starting.
I20260806 06:43:54.885092 28263 tablet_bootstrap.cc:654] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:54.885561 28264 tablet_bootstrap.cc:654] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:54.885545 28263 log.cc:824] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:54.886024 28264 log.cc:824] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:54.886333 28263 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: No bootstrap required, opened a new log
I20260806 06:43:54.886404 28263 ts_tablet_manager.cc:1397] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:43:54.886595 28261 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: No bootstrap required, opened a new log
I20260806 06:43:54.886654 28261 ts_tablet_manager.cc:1397] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:43:54.889343 28261 raft_consensus.cc:348] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.889533 28261 raft_consensus.cc:374] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:54.889605 28261 raft_consensus.cc:729] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc319634c54f439185fea0ca4062a4f0, State: Initialized, Role: FOLLOWER
I20260806 06:43:54.889489 28263 raft_consensus.cc:348] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.889755 28261 consensus_queue.cc:260] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.889881 28263 raft_consensus.cc:374] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:54.889976 28263 raft_consensus.cc:729] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230e21c9a3ea466e9cf17b6f3f9db3ac, State: Initialized, Role: FOLLOWER
I20260806 06:43:54.890051 28261 ts_tablet_manager.cc:1428] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:43:54.890200 27957 heartbeater.cc:503] Master 127.14.51.126:43713 was elected leader, sending a full tablet report...
I20260806 06:43:54.891177 28264 tablet_bootstrap.cc:492] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: No bootstrap required, opened a new log
I20260806 06:43:54.891245 28264 ts_tablet_manager.cc:1397] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 06:43:54.891623 28263 consensus_queue.cc:260] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.891993 28263 ts_tablet_manager.cc:1428] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260806 06:43:54.892144 28228 heartbeater.cc:503] Master 127.14.51.126:43713 was elected leader, sending a full tablet report...
I20260806 06:43:54.894744 28264 raft_consensus.cc:348] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.895023 28264 raft_consensus.cc:374] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:54.895136 28264 raft_consensus.cc:729] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589fa9205898498aa8e156dc244e13ef, State: Initialized, Role: FOLLOWER
I20260806 06:43:54.895263 28264 consensus_queue.cc:260] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:54.895526 28264 ts_tablet_manager.cc:1428] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260806 06:43:54.895591 28094 heartbeater.cc:503] Master 127.14.51.126:43713 was elected leader, sending a full tablet report...
W20260806 06:43:54.936301 27958 tablet.cc:2369] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:43:54 dist-test-slave-1xrh krb5kdc[26743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for HTTP/127.14.51.84@KRBTEST.COM
W20260806 06:43:55.075137 28229 tablet.cc:2369] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:43:55.090385 28269 raft_consensus.cc:482] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:43:55.090547 28269 raft_consensus.cc:504] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:55.091060 28269 leader_election.cc:288] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 230e21c9a3ea466e9cf17b6f3f9db3ac (127.14.51.67:44671), fc319634c54f439185fea0ca4062a4f0 (127.14.51.65:33001)
I20260806 06:43:55.095683 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.091514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:47271 (local address 127.14.51.67:44671)
0806 06:43:55.091563 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:55.091565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:55.091704 (+ 139us) server_negotiation.cc:407] Connection header received
0806 06:43:55.091779 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:55.091781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:55.091791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:55.091844 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:55.092203 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.092977 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.094622 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.095202 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.095254 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:55.095263 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:55.095580 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:43:55.095618 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:43:55.096261 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.091227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:44671 (local address 127.14.51.66:47271)
0806 06:43:55.091616 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:55.091627 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:55.091713 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:55.091896 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:55.091896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:55.091899 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:55.092095 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:55.092098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.093087 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.093088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:55.094519 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:55.094522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.095415 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.095417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:55.095496 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:55.095499 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:43:55.095987 (+ 488us) client_negotiation.cc:240] Negotiation successful
0806 06:43:55.096037 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":73,"thread_start_us":117,"threads_started":1}
I20260806 06:43:55.097015 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cc474d50544d8f965c4f5a0cc578e8" candidate_uuid: "589fa9205898498aa8e156dc244e13ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" is_pre_election: true
I20260806 06:43:55.097183 28182 raft_consensus.cc:2454] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589fa9205898498aa8e156dc244e13ef in term 0.
I20260806 06:43:55.097466 27980 leader_election.cc:302] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230e21c9a3ea466e9cf17b6f3f9db3ac, 589fa9205898498aa8e156dc244e13ef; no voters:
I20260806 06:43:55.097601 28269 raft_consensus.cc:2781] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:43:55.097642 28269 raft_consensus.cc:482] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:43:55.097663 28269 raft_consensus.cc:3037] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:55.097666 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.091477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44467 (local address 127.14.51.65:33001)
0806 06:43:55.091537 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:55.091539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:55.091875 (+ 336us) server_negotiation.cc:407] Connection header received
0806 06:43:55.091992 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:55.091993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:55.092003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:55.092058 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:43:55.092345 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.093031 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.096093 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.096753 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.096810 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:55.096816 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:55.097552 (+ 736us) server_negotiation.cc:299] Negotiation successful
0806 06:43:55.097599 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:43:55.097929 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.091248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:33001 (local address 127.14.51.66:44467)
0806 06:43:55.091822 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:55.091831 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:55.091897 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:55.092129 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:55.092129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:55.092131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:43:55.092266 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:55.092267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.093125 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.093126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:55.095993 (+ 2867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:55.095996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.097350 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.097351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:55.097459 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:55.097464 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:43:55.097762 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 06:43:55.097823 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":1562,"thread_start_us":135,"threads_started":1}
I20260806 06:43:55.098268 27911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cc474d50544d8f965c4f5a0cc578e8" candidate_uuid: "589fa9205898498aa8e156dc244e13ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc319634c54f439185fea0ca4062a4f0" is_pre_election: true
I20260806 06:43:55.098405 27911 raft_consensus.cc:2454] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589fa9205898498aa8e156dc244e13ef in term 0.
I20260806 06:43:55.098778 28269 raft_consensus.cc:504] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:55.098889 28269 leader_election.cc:288] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [CANDIDATE]: Term 1 election: Requested vote from peers 230e21c9a3ea466e9cf17b6f3f9db3ac (127.14.51.67:44671), fc319634c54f439185fea0ca4062a4f0 (127.14.51.65:33001)
I20260806 06:43:55.099081 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cc474d50544d8f965c4f5a0cc578e8" candidate_uuid: "589fa9205898498aa8e156dc244e13ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac"
I20260806 06:43:55.099213 28182 raft_consensus.cc:3037] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:55.100385 28182 raft_consensus.cc:2454] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589fa9205898498aa8e156dc244e13ef in term 1.
I20260806 06:43:55.100608 27980 leader_election.cc:302] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230e21c9a3ea466e9cf17b6f3f9db3ac, 589fa9205898498aa8e156dc244e13ef; no voters:
I20260806 06:43:55.100755 27911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cc474d50544d8f965c4f5a0cc578e8" candidate_uuid: "589fa9205898498aa8e156dc244e13ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc319634c54f439185fea0ca4062a4f0"
I20260806 06:43:55.100855 27911 raft_consensus.cc:3037] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:55.101696 27911 raft_consensus.cc:2454] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589fa9205898498aa8e156dc244e13ef in term 1.
I20260806 06:43:55.101894 28269 raft_consensus.cc:2781] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:43:55.102092 28269 raft_consensus.cc:686] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 1 LEADER]: Becoming Leader. State: Replica: 589fa9205898498aa8e156dc244e13ef, State: Running, Role: LEADER
I20260806 06:43:55.102195 28269 consensus_queue.cc:237] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } }
I20260806 06:43:55.103053 27756 catalog_manager.cc:5495] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 589fa9205898498aa8e156dc244e13ef (127.14.51.66). New cstate: current_term: 1 leader_uuid: "589fa9205898498aa8e156dc244e13ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "589fa9205898498aa8e156dc244e13ef" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45385 } health_report { overall_health: HEALTHY } } }
W20260806 06:43:55.117609 28095 tablet.cc:2369] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:43:55.150175 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.147176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.14.51.66:45385)
0806 06:43:55.147272 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:55.147275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:55.147283 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:43:55.147332 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:55.147333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:55.147354 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:55.147421 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:43:55.147602 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.148379 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.149229 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:55.149577 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:55.149659 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:55.149822 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:43:55.149932 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:43:55.150019 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:55.150063 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:43:55.150084 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 06:43:55.152681 27911 raft_consensus.cc:1261] T b2cc474d50544d8f965c4f5a0cc578e8 P fc319634c54f439185fea0ca4062a4f0 [term 1 FOLLOWER]: Refusing update from remote peer 589fa9205898498aa8e156dc244e13ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:55.153092 28269 consensus_queue.cc:1035] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc319634c54f439185fea0ca4062a4f0" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:55.155062 28182 raft_consensus.cc:1261] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 1 FOLLOWER]: Refusing update from remote peer 589fa9205898498aa8e156dc244e13ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:55.155433 28269 consensus_queue.cc:1035] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "230e21c9a3ea466e9cf17b6f3f9db3ac" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:55.158651 28277 mvcc.cc:205] Tried to move back new op lower bound from 7315450409582059520 to 7315450409379053568. Current Snapshot: MvccSnapshot[applied={T|T < 7315450409582059520}]
I20260806 06:43:55.159103 28280 mvcc.cc:205] Tried to move back new op lower bound from 7315450409582059520 to 7315450409379053568. Current Snapshot: MvccSnapshot[applied={T|T < 7315450409582059520}]
I20260806 06:43:55.162962 27756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:43:55.163045 27756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:43:55.163746 28278 mvcc.cc:205] Tried to move back new op lower bound from 7315450409582059520 to 7315450409379053568. Current Snapshot: MvccSnapshot[applied={T|T < 7315450409582059520 or (T in {7315450409582059520})}]
I20260806 06:43:55.163901 27756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06eac8bec3634abd9ce4e75c5132ce71: Sending DeleteTablet for 3 replicas of tablet b2cc474d50544d8f965c4f5a0cc578e8
I20260806 06:43:55.164538 28162 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:43:55 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:43988
I20260806 06:43:55.164893 27873 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:43:55 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:60108
I20260806 06:43:55.165174 28027 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cc474d50544d8f965c4f5a0cc578e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:43:55 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:32906
I20260806 06:43:55.165762 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27796
I20260806 06:43:55.175004 28288 tablet_replica.cc:330] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: stopping tablet replica
I20260806 06:43:55.175161 28286 tablet_replica.cc:330] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: stopping tablet replica
I20260806 06:43:55.175312 28288 raft_consensus.cc:2229] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:43:55.175450 28286 raft_consensus.cc:2229] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:43:55.175638 28288 raft_consensus.cc:2258] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:43:55.175894 28286 raft_consensus.cc:2258] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:43:55.176462 28288 ts_tablet_manager.cc:1905] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:43:55.176884 28286 ts_tablet_manager.cc:1905] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:43:55.179925 28286 ts_tablet_manager.cc:1918] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:43:55.180558 28288 ts_tablet_manager.cc:1918] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:43:55.180684 28286 log.cc:1196] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/b2cc474d50544d8f965c4f5a0cc578e8
I20260806 06:43:55.180797 28288 log.cc:1196] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/b2cc474d50544d8f965c4f5a0cc578e8
I20260806 06:43:55.181162 28286 ts_tablet_manager.cc:1939] T b2cc474d50544d8f965c4f5a0cc578e8 P 230e21c9a3ea466e9cf17b6f3f9db3ac: Deleting consensus metadata
I20260806 06:43:55.181497 28288 ts_tablet_manager.cc:1939] T b2cc474d50544d8f965c4f5a0cc578e8 P 589fa9205898498aa8e156dc244e13ef: Deleting consensus metadata
I20260806 06:43:55.181942 27742 catalog_manager.cc:4841] TS 230e21c9a3ea466e9cf17b6f3f9db3ac (127.14.51.67:44671): tablet b2cc474d50544d8f965c4f5a0cc578e8 (table test-table [id=98e2f1271c294dc8a6b2d665024ef516]) successfully deleted
I20260806 06:43:55.182446 27743 catalog_manager.cc:4841] TS 589fa9205898498aa8e156dc244e13ef (127.14.51.66:45385): tablet b2cc474d50544d8f965c4f5a0cc578e8 (table test-table [id=98e2f1271c294dc8a6b2d665024ef516]) successfully deleted
W20260806 06:43:55.182854 27741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:33001 (error 108)
I20260806 06:43:55.183718 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27961
I20260806 06:43:55.183730 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:55.183589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:33001 (local address 127.0.0.1:60124)
0806 06:43:55.183640 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:55.183695 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:33001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:43:55.188117 27741 catalog_manager.cc:4584] TS fc319634c54f439185fea0ca4062a4f0 (127.14.51.65:33001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2cc474d50544d8f965c4f5a0cc578e8: Network error: Client connection negotiation failed: client connection to 127.14.51.65:33001: connect: Connection refused (error 111)
I20260806 06:43:55.191586 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28097
I20260806 06:43:55.196785 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 27724
I20260806 06:43:55.203570 14541 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:43:55.547646 14541 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:43:55.547741 14541 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:43:55.892153 14541 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:43:55.892 UTC [26773] LOG: received smart shutdown request
2026-08-06 06:43:55.896 UTC [26773] LOG: background worker "logical replication launcher" (PID 26782) exited with exit code 1
2026-08-06 06:43:55.896 UTC [26777] LOG: shutting down
2026-08-06 06:43:56.055 UTC [26773] LOG: database system is shut down
2026-08-06T06:43:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47528 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:43:56 dist-test-slave-1xrh krb5kdc[28298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:43:56 dist-test-slave-1xrh krb5kdc[28298](info): set up 2 sockets
Aug 06 06:43:56 dist-test-slave-1xrh krb5kdc[28298](info): commencing operation
krb5kdc: starting...
W20260806 06:43:57.217710 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:43:57.334626 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33093
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:43:57.413888 28311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:57.414077 28311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:57.414099 28311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:57.415163 28311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:57.415196 28311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:57.415210 28311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:57.415223 28311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:57.415235 28311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:57.416653 28311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:57.416826 28311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:57.416980 28311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:57.421803 28316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:57.421778 28317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:57.421859 28311 server_base.cc:1034] running on GCE node
W20260806 06:43:57.421778 28319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:57.422127 28311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:57.422457 28311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:57.423604 28311 hybrid_clock.cc:648] HybridClock initialized: now 1785998637423600 us; error 38 us; skew 500 ppm
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:57.426051 28311 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:43:57.426422 28311 webserver.cc:460] Webserver started at http://127.14.51.126:43005/ using document root <none> and password file <none>
I20260806 06:43:57.426614 28311 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:57.426653 28311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:57.426738 28311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:43:57.427784 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "7a974474493f4821bbae6552df1fa75a"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "b1ed200f86cadaf247df243a1ec0af3b"
server_key_iv: "4494dc7dae29399844a57e7ec3a1c458"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.428085 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "7a974474493f4821bbae6552df1fa75a"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "b1ed200f86cadaf247df243a1ec0af3b"
server_key_iv: "4494dc7dae29399844a57e7ec3a1c458"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.429499 28311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:43:57.430200 28326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:57.430371 28311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:43:57.430444 28311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "7a974474493f4821bbae6552df1fa75a"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "b1ed200f86cadaf247df243a1ec0af3b"
server_key_iv: "4494dc7dae29399844a57e7ec3a1c458"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.430510 28311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:57.449309 28311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:57.451521 28311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:57.451683 28311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:57.456696 28311 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:33093
I20260806 06:43:57.456733 28387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:33093 every 8 connection(s)
I20260806 06:43:57.457120 28311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:43:57.457986 28388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:57.458992 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28311
I20260806 06:43:57.459105 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:43:57.459358 14541 external_mini_cluster.cc:1428] Setting key 9bc70a25ace0f0d86df50e1034ea8511
I20260806 06:43:57.459790 28388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.125@KRBTEST.COM; defaulting to no policy
I20260806 06:43:57.470539 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.460606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.14.51.126:33093)
0806 06:43:57.461023 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.461028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.461060 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:43:57.461603 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.461605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.461622 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.461702 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.462399 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.463275 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.464674 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.464982 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.465084 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.467326 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.467335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.467335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.467342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.469018 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.469369 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.469372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.469372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.469468 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.469756 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.469757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.469757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.469972 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.470049 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.470227 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.470377 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":380,"thread_start_us":60,"threads_started":1}
I20260806 06:43:57.470561 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.460539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33093 (local address 127.0.0.1:51810)
0806 06:43:57.460732 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.460779 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.461434 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.461737 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.461740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.462095 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.462293 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.462298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.463375 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.463376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.464585 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.464589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.465094 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.465095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.465184 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.465646 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.465657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.467234 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.469094 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.469100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.469102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.469295 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.469549 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.469550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.469550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.469659 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.470054 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.470059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.470114 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.470187 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.470303 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.125@KRBTEST.COM" created.
I20260806 06:43:57.471241 28388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:43:57.471566 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.471457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48714)
0806 06:43:57.471492 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.471532 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 06:43:57.471735 28328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
I20260806 06:43:57.472160 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.472079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48718)
0806 06:43:57.472121 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.472145 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:43:57.472282 28388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:36361: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
I20260806 06:43:57.486356 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:36361
--webserver_interface=127.14.51.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:43:57.521485 28388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } attempt: 1
I20260806 06:43:57.521938 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.521812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48720)
0806 06:43:57.521879 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.521914 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 06:43:57.522264 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.522178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48734)
0806 06:43:57.522224 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.522248 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:43:57.522369 28388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:36361: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
W20260806 06:43:57.565124 28395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:57.565307 28395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:57.565338 28395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:57.566459 28395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:57.566501 28395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:57.566516 28395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:57.566565 28395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:57.566593 28395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:57.568670 28395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.51.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:57.568904 28395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:57.569120 28395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:43:57.573702 28401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:57.573706 28403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:57.573717 28400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:57.573907 28395 server_base.cc:1034] running on GCE node
I20260806 06:43:57.574139 28395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:57.574409 28395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:57.575548 28395 hybrid_clock.cc:648] HybridClock initialized: now 1785998637575537 us; error 31 us; skew 500 ppm
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:57.578001 28395 init.cc:377] Logged in from keytab as kudu/127.14.51.125@KRBTEST.COM (short username kudu)
I20260806 06:43:57.578393 28395 webserver.cc:460] Webserver started at http://127.14.51.125:42939/ using document root <none> and password file <none>
I20260806 06:43:57.578619 28395 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:57.578687 28395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:57.578835 28395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:43:57.579864 28395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data/instance:
uuid: "01e44bf5b35e4b258573beff174871c3"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "e4789b94fc33b4e86fe2e498e8dc8c25"
server_key_iv: "83665feee48d00c91d9c8c138a02141b"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.580212 28395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal/instance:
uuid: "01e44bf5b35e4b258573beff174871c3"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "e4789b94fc33b4e86fe2e498e8dc8c25"
server_key_iv: "83665feee48d00c91d9c8c138a02141b"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.581488 28395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:43:57.582203 28410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:57.582391 28395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:43:57.582504 28395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "01e44bf5b35e4b258573beff174871c3"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "e4789b94fc33b4e86fe2e498e8dc8c25"
server_key_iv: "83665feee48d00c91d9c8c138a02141b"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.582597 28395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:57.590523 28388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } attempt: 2
I20260806 06:43:57.590903 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.590779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48746)
0806 06:43:57.590838 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.590874 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:43:57.591284 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.591201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48752)
0806 06:43:57.591242 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.591266 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:43:57.591401 28388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:36361: Network error: Client connection negotiation failed: client connection to 127.14.51.125:36361: connect: Connection refused (error 111)
I20260806 06:43:57.595604 28395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:57.597767 28395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:57.597919 28395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:57.601770 28395 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.125:36361
I20260806 06:43:57.601876 28462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.125:36361 every 8 connection(s)
I20260806 06:43:57.602149 28395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
I20260806 06:43:57.602913 28463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:57.604421 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:57.610484 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28395
I20260806 06:43:57.610599 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-1/wal/instance
I20260806 06:43:57.610793 14541 external_mini_cluster.cc:1428] Setting key ce52b1bed6199ec245c8ceb2c2f6a60f
I20260806 06:43:57.613021 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.605075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.14.51.126:33093)
0806 06:43:57.605167 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.605169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.605347 (+ 178us) server_negotiation.cc:407] Connection header received
0806 06:43:57.605779 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.605780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.605793 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.605848 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.606456 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.606932 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.607933 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.608130 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.608213 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.610103 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.610105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.610105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.610111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.611579 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.612088 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.612090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.612090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.612158 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.612415 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.612416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.612416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.612556 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.612628 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.612843 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.612932 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:43:57.613101 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.605035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33093 (local address 127.0.0.1:51826)
0806 06:43:57.605220 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.605250 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.605679 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.605956 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.605959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.606185 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.606359 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.606363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.607026 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.607027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.607843 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.607845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.608224 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.608225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.608339 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.608717 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.608725 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.610019 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.611664 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.611671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.611673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.611988 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.612236 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.612237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.612237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.612349 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.612621 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.612626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.612723 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.612797 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.612940 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260806 06:43:57.613638 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.124@KRBTEST.COM; defaulting to no policy
I20260806 06:43:57.620759 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.613880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48764)
0806 06:43:57.613919 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.613928 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.613966 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.614233 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.614233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.614377 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.614417 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.614419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.615335 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.615336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.615805 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.615806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.616273 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.616274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.616340 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.616722 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.616726 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.617870 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.619753 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.619754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.619754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.619913 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.620127 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.620128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.620128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.620180 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.620470 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.620471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.620506 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.620564 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.620655 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:43:57.620834 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.613950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.14.51.125:36361)
0806 06:43:57.614102 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.614106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.614120 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:43:57.614135 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.614136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.614146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.614220 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.614490 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.615240 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.615890 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.616188 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.616264 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.617963 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.617971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.617971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.617977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.619652 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.620002 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.620005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.620005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.620059 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.620247 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.620248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.620248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.620399 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.620490 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.620600 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.620704 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.124@KRBTEST.COM" created.
I20260806 06:43:57.621363 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:43:57.621672 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.621572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58258)
0806 06:43:57.621612 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.621653 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:43:57.622792 28413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
I20260806 06:43:57.623246 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.623169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58270)
0806 06:43:57.623201 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.623225 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 06:43:57.623378 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:39439: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
I20260806 06:43:57.635896 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:39439
--webserver_interface=127.14.51.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:43:57.672581 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } attempt: 1
I20260806 06:43:57.673103 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.672932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58282)
0806 06:43:57.673004 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.673063 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:43:57.673487 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.673399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58286)
0806 06:43:57.673441 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.673469 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:43:57.673611 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:39439: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
I20260806 06:43:57.682590 28388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } attempt: 3
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:43:57.689880 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.683002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.14.51.125:36361)
0806 06:43:57.683067 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.683069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.683102 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:43:57.683146 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.683147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.683159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.683236 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.683587 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.684144 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.684905 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.685113 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.685209 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.687264 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.687265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.687266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.687273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.688715 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.689090 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.689092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.689092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.689152 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.689368 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.689369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.689369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.689478 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.689555 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.689718 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.689788 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:43:57.689893 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.682949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48780)
0806 06:43:57.683001 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.683013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.683069 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.683236 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.683236 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.683425 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.683493 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.683495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.684252 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.684254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.684825 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.684828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.685221 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.685222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.685314 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.685749 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.685755 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.687176 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.688827 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.688828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.688828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.688995 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.689224 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.689225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.689225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.689304 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.689547 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.689548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.689591 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.689691 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.689788 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:43:57.690289 28388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:43:57.690601 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.690503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58302)
0806 06:43:57.690555 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.690583 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:43:57.690939 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.690825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58308)
0806 06:43:57.690889 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.690913 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 06:43:57.691028 28388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:39439: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
W20260806 06:43:57.716573 28470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:43:57.716782 28470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:43:57.716818 28470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:43:57.717937 28470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:43:57.718003 28470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:43:57.718027 28470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:43:57.718055 28470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:43:57.718101 28470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:43:57.719713 28470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33093,127.14.51.125:36361,127.14.51.124:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:39439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.51.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:43:57.719971 28470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:43:57.720184 28470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:43:57.722154 28388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } attempt: 1
I20260806 06:43:57.722581 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.722435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58324)
0806 06:43:57.722500 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.722548 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:43:57.722947 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.722851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58332)
0806 06:43:57.722900 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.722927 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:43:57.723068 28388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:39439: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
W20260806 06:43:57.725099 28478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:57.725135 28476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:43:57.725099 28475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:43:57.725519 28470 server_base.cc:1034] running on GCE node
I20260806 06:43:57.725757 28470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:43:57.726069 28470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:43:57.727182 28470 hybrid_clock.cc:648] HybridClock initialized: now 1785998637727157 us; error 34 us; skew 500 ppm
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:43:57.729676 28470 init.cc:377] Logged in from keytab as kudu/127.14.51.124@KRBTEST.COM (short username kudu)
I20260806 06:43:57.730144 28470 webserver.cc:460] Webserver started at http://127.14.51.124:35651/ using document root <none> and password file <none>
I20260806 06:43:57.730381 28470 fs_manager.cc:359] Metadata directory not provided
I20260806 06:43:57.730445 28470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:43:57.730587 28470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:43:57.731720 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data/instance:
uuid: "8a1b9dc7db8a49388ac58355c626aca1"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "3e7b1eaed8515f378d38e529ee59c7fe"
server_key_iv: "a87606610cf2e4896788b431cc8b42e4"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.732071 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal/instance:
uuid: "8a1b9dc7db8a49388ac58355c626aca1"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "3e7b1eaed8515f378d38e529ee59c7fe"
server_key_iv: "a87606610cf2e4896788b431cc8b42e4"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.733477 28470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:43:57.734319 28485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:43:57.734508 28470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:43:57.734602 28470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "8a1b9dc7db8a49388ac58355c626aca1"
format_stamp: "Formatted at 2026-08-06 06:43:57 on dist-test-slave-1xrh"
server_key: "3e7b1eaed8515f378d38e529ee59c7fe"
server_key_iv: "a87606610cf2e4896788b431cc8b42e4"
server_key_version: "encryptionkey@0"
I20260806 06:43:57.734685 28470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:43:57.741779 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } attempt: 2
I20260806 06:43:57.742180 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.742052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58342)
0806 06:43:57.742108 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.742146 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:43:57.742516 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.742425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58354)
0806 06:43:57.742469 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.742495 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:43:57.742640 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:39439: Network error: Client connection negotiation failed: client connection to 127.14.51.124:39439: connect: Connection refused (error 111)
I20260806 06:43:57.753288 28470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:43:57.754977 28470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:43:57.755160 28470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:43:57.758920 28470 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.124:39439
I20260806 06:43:57.758980 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.124:39439 every 8 connection(s)
I20260806 06:43:57.759333 28470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
I20260806 06:43:57.759987 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28470
I20260806 06:43:57.760164 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998482572191-14541-0/minicluster-data/master-2/wal/instance
I20260806 06:43:57.760226 28541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:43:57.760385 14541 external_mini_cluster.cc:1428] Setting key 14513484f27b751da712cf03c473edd4
I20260806 06:43:57.762523 28541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:43:57.770800 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.763218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.14.51.126:33093)
0806 06:43:57.763280 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.763282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.763428 (+ 146us) server_negotiation.cc:407] Connection header received
0806 06:43:57.763461 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.763462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.763472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.763538 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.763955 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.764423 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.765183 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.765402 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.765478 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.767461 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.767463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.767464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.767471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.769251 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.769720 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.769722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.769722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.769784 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.770052 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.770053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.770053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.770206 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.770274 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.770614 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.770699 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:43:57.771006 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.763166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33093 (local address 127.0.0.1:51828)
0806 06:43:57.763346 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.763371 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.763424 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.763552 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.763554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.763767 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.763863 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.763865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.764510 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.764511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.765094 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.765096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.765485 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.765487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.765561 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.765980 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.765989 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.767369 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.769349 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.769355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.769357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.769641 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.769867 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.769868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.769868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.769975 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.770307 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.770314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.770464 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.770573 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.770718 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260806 06:43:57.771009 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.760896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.14.51.124:39439)
0806 06:43:57.761093 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.761102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.761135 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:43:57.761804 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.761814 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.761874 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.762023 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.762566 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.763779 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.764540 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.764950 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.765078 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.768025 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.768031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.768031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.768038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.769762 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.770140 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.770143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.770143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.770201 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.770405 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.770406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.770406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.770535 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.770680 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.770754 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.770841 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260806 06:43:57.771519 28541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:43:57.779078 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.771756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:36361 (local address 127.0.0.1:48784)
0806 06:43:57.771917 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.771946 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.771991 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.772101 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.772102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.772261 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.772301 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.772303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.773249 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.773251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.774110 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.774112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.774613 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.774614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.774687 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.775156 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.775161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.776414 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.778064 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.778065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.778065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.778227 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.778418 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.778419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.778420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.778474 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.778784 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.778784 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.778819 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.778872 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.778963 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 06:43:57.779085 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.771806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.14.51.125:36361)
0806 06:43:57.771846 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.771848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.772002 (+ 154us) server_negotiation.cc:407] Connection header received
0806 06:43:57.772047 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.772048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.772057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.772095 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.772370 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.773089 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.774234 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.774532 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.774599 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.776511 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.776513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.776513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.776520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.777956 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.778297 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.778298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.778298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.778353 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.778544 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.778546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.778546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.778709 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.778778 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.778923 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.779006 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:43:57.779474 28541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:43:57.779872 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.771798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.14.51.126:33093)
0806 06:43:57.771831 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.771833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.771853 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:43:57.771901 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.771901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.771909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.771948 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.772538 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.773153 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.774235 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.774529 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.774599 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.777182 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.777183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.777183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.777190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.778584 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.778978 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.778979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.778979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.779032 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.779290 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.779292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.779292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.779436 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.779631 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.779674 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.779753 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:43:57.786897 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.779735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58358)
0806 06:43:57.779780 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.779789 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.779821 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.779971 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.779972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.780095 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.780116 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.780117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.780988 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.780989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.781691 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.781694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.782159 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.782160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.782212 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.782560 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.782565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.783986 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.785647 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.785648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.785648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.785856 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.786116 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.786116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.786117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.786180 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.786515 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.786516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.786557 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.786622 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.786735 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:43:57.786952 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.779804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.14.51.124:39439)
0806 06:43:57.779840 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.779841 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.779851 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:43:57.779869 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.779870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.779878 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.779965 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.780195 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.780763 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.781762 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.782061 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.782134 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.784095 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.784096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.784096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.784102 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.785573 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.785952 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.785952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.785953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.786025 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.786265 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.786266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.786266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.786405 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.786480 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.786674 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.786784 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":18}
I20260806 06:43:57.788254 28541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: Bootstrap starting.
I20260806 06:43:57.788928 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.780822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.14.51.125:36361)
0806 06:43:57.780860 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.780862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.780871 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:43:57.780899 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.780900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.780908 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.780956 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.781509 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.782088 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.782990 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.783238 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.783293 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.785850 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.785851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.785852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.785858 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.787779 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.788126 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.788127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.788127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.788181 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.788406 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.788407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.788407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.788516 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.788719 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.788755 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.788832 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:43:57.789198 28541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:57.789522 28541 log.cc:824] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:57.790365 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:43:57.790385 28541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: No bootstrap required, opened a new log
I20260806 06:43:57.792184 28541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.792342 28541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:57.792407 28541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1b9dc7db8a49388ac58355c626aca1, State: Initialized, Role: FOLLOWER
I20260806 06:43:57.792549 28541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.792857 28541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 06:43:57.794873 28565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:43:57.795017 28565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:43:57.796036 28554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.796175 28554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:43:57.794870 28470 master_runner.cc:186] Error getting master registration for 127.14.51.126:33093: OK, instance_id { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" instance_seqno: 1785998637455285 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:43:57.798113 28470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:43:57.798254 28388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } attempt: 2
I20260806 06:43:57.800611 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.791913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.14.51.124:39439)
0806 06:43:57.791956 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.791957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.792119 (+ 162us) server_negotiation.cc:407] Connection header received
0806 06:43:57.792173 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.792174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.792183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.792245 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.792853 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.793582 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.794722 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.795019 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.795072 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.796697 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.796698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.796699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.796706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.798471 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.798848 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.798849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.798850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.798911 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.799461 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.799462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.799462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.799615 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.799704 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.800134 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.800238 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:43:57.800721 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.792051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.14.51.125:36361)
0806 06:43:57.792085 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.792087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.792852 (+ 765us) server_negotiation.cc:407] Connection header received
0806 06:43:57.792943 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.792944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.792953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.792980 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.793664 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.794339 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.795311 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.795636 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.795678 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.797044 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.797046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.797046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.797053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.798696 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.799284 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.799285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.799286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.799341 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.799541 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.799542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.799542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.799653 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.800304 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.800532 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.800612 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:43:57.802960 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.791723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.14.51.126:33093)
0806 06:43:57.791771 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.791773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.791882 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:43:57.791985 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.791985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.791996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.792028 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.792767 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.793473 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.798332 (+ 4859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.798639 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.798678 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.800142 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.800144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.800144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.800150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.801732 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.802099 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.802100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.802100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.802159 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.802390 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.802391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.802392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.802517 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.802617 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.802778 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.802846 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:43:57.806167 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.798483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58378)
0806 06:43:57.798527 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.798535 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.798570 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.799167 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.799168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.799342 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.799382 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.799383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.800380 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.800381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.801118 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.801121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.801681 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.801682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.801739 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.802119 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.802123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.803524 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.805289 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.805290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.805291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.805468 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.805673 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.805673 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.805674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.805723 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.805970 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.805971 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.806001 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.806052 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.806112 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:43:57.806274 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.798792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.14.51.124:39439)
0806 06:43:57.798990 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.798992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.799001 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:43:57.799021 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.799022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.799031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.799463 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.799496 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.800284 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.801219 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.801573 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.801671 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.803635 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.803636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.803637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.803643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.805206 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.805522 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.805523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.805523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.805606 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.805782 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.805783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.805783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.805906 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.805974 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.806085 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.806163 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
I20260806 06:43:57.807857 28388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: Bootstrap starting.
I20260806 06:43:57.808595 28388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:57.808914 28388 log.cc:824] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:57.809979 28388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: No bootstrap required, opened a new log
I20260806 06:43:57.812103 28388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.812237 28388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:57.812275 28388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a974474493f4821bbae6552df1fa75a, State: Initialized, Role: FOLLOWER
I20260806 06:43:57.812414 28388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.812729 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.812762 28388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:43:57.812831 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:43:57.814639 28580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:43:57.814735 28580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:43:57.815371 28311 master_runner.cc:186] Error getting master registration for 127.14.51.125:36361: OK, instance_id { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" instance_seqno: 1785998637600738 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:43:57.817765 28311 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:43:57.833794 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } attempt: 3
Aug 06 06:43:57 dist-test-slave-1xrh krb5kdc[28298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:43:57.842109 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.834151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:39439 (local address 127.0.0.1:58382)
0806 06:43:57.834203 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:43:57.834214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:43:57.834271 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:43:57.834444 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:43:57.834445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:43:57.834606 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:43:57.834646 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.834648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.836115 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.836116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.836661 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:43:57.836663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.837136 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.837137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:43:57.837194 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.837678 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:43:57.837684 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:43:57.839061 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:43:57.840729 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.840730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.840730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.840971 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.841453 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:43:57.841454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:43:57.841454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:43:57.841516 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:43:57.841781 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:43:57.841782 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:43:57.841827 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:43:57.841915 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:43:57.842009 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:43:57.842109 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:43:57.834179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.14.51.124:39439)
0806 06:43:57.834217 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:43:57.834220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:43:57.834271 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:43:57.834350 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:43:57.834351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:43:57.834362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:43:57.834439 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:43:57.834723 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.836022 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.836756 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:43:57.837027 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:43:57.837138 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:43:57.839162 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:43:57.839163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:43:57.839163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:43:57.839171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:43:57.840623 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.841290 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.841290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.841291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.841364 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:43:57.841583 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:43:57.841584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:43:57.841584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:43:57.841711 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:43:57.841790 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:43:57.841939 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:43:57.842025 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:43:57.843794 28463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: Bootstrap starting.
I20260806 06:43:57.844424 28463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: Neither blocks nor log segments found. Creating new log.
I20260806 06:43:57.844725 28463 log.cc:824] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:43:57.845471 28463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: No bootstrap required, opened a new log
I20260806 06:43:57.846863 28463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.846994 28463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:43:57.847072 28463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e44bf5b35e4b258573beff174871c3, State: Initialized, Role: FOLLOWER
I20260806 06:43:57.847254 28463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.847581 28583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.847642 28463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:43:57.847718 28583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:43:57.848465 28416 tablet.cc:2369] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:43:57.849553 28594 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:43:57.849689 28594 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:43:57.850534 28583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:43:57.850598 28583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.850818 28583 leader_election.cc:288] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a974474493f4821bbae6552df1fa75a (127.14.51.126:33093), 8a1b9dc7db8a49388ac58355c626aca1 (127.14.51.124:39439)
I20260806 06:43:57.851226 28514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e44bf5b35e4b258573beff174871c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1b9dc7db8a49388ac58355c626aca1" is_pre_election: true
I20260806 06:43:57.851398 28514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01e44bf5b35e4b258573beff174871c3 in term 0.
I20260806 06:43:57.851606 28413 leader_election.cc:302] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01e44bf5b35e4b258573beff174871c3, 8a1b9dc7db8a49388ac58355c626aca1; no voters:
I20260806 06:43:57.852272 28583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:43:57.852346 28583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:43:57.852377 28583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:57.853240 28583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.853369 28583 leader_election.cc:288] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [CANDIDATE]: Term 1 election: Requested vote from peers 7a974474493f4821bbae6552df1fa75a (127.14.51.126:33093), 8a1b9dc7db8a49388ac58355c626aca1 (127.14.51.124:39439)
I20260806 06:43:57.853940 28361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e44bf5b35e4b258573beff174871c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a974474493f4821bbae6552df1fa75a"
I20260806 06:43:57.854065 28361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:57.854970 28361 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01e44bf5b35e4b258573beff174871c3 in term 1.
I20260806 06:43:57.856174 28412 leader_election.cc:302] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01e44bf5b35e4b258573beff174871c3, 7a974474493f4821bbae6552df1fa75a; no voters:
I20260806 06:43:57.856398 28395 master_runner.cc:263] No leader master found from master 01e44bf5b35e4b258573beff174871c3
I20260806 06:43:57.856442 28395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:43:57.856547 28514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e44bf5b35e4b258573beff174871c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1b9dc7db8a49388ac58355c626aca1"
I20260806 06:43:57.856619 28514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:43:57.857507 28514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01e44bf5b35e4b258573beff174871c3 in term 1.
I20260806 06:43:57.857823 28362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e44bf5b35e4b258573beff174871c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a974474493f4821bbae6552df1fa75a" is_pre_election: true
I20260806 06:43:57.857913 28362 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01e44bf5b35e4b258573beff174871c3 in term 1. Re-sending same reply.
I20260806 06:43:57.858165 28583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:43:57.858300 28583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [term 1 LEADER]: Becoming Leader. State: Replica: 01e44bf5b35e4b258573beff174871c3, State: Running, Role: LEADER
I20260806 06:43:57.858390 28583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } }
I20260806 06:43:57.858731 28583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e44bf5b35e4b258573beff174871c3. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.858795 28583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:57.859793 28598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:43:57.860052 28598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:43:57.865947 28514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [term 1 FOLLOWER]: Refusing update from remote peer 01e44bf5b35e4b258573beff174871c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:57.866200 28362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [term 1 FOLLOWER]: Refusing update from remote peer 01e44bf5b35e4b258573beff174871c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:43:57.866655 28583 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:57.866792 28583 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:43:57.868201 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e44bf5b35e4b258573beff174871c3. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.868299 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:43:57.869113 28554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e44bf5b35e4b258573beff174871c3. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.869205 28554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:43:57.870745 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.870852 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:43:57.871001 28583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.871078 28583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:57.871174 28599 mvcc.cc:205] Tried to move back new op lower bound from 7315450420695932928 to 7315450420668469248. Current Snapshot: MvccSnapshot[applied={T|T < 7315450420695932928}]
I20260806 06:43:57.871275 28600 mvcc.cc:205] Tried to move back new op lower bound from 7315450420695932928 to 7315450420668469248. Current Snapshot: MvccSnapshot[applied={T|T < 7315450420695932928}]
I20260806 06:43:57.872236 28596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.872346 28596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:43:57.872740 28554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01e44bf5b35e4b258573beff174871c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a974474493f4821bbae6552df1fa75a" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33093 } } peers { permanent_uuid: "01e44bf5b35e4b258573beff174871c3" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 36361 } } peers { permanent_uuid: "8a1b9dc7db8a49388ac58355c626aca1" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 39439 } } }
I20260806 06:43:57.872844 28554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:43:57.872917 28603 mvcc.cc:205] Tried to move back new op lower bound from 7315450420695932928 to 7315450420668469248. Current Snapshot: MvccSnapshot[applied={T|T < 7315450420695932928}]
I20260806 06:43:57.873456 28598 catalog_manager.cc:1347] Generated new cluster ID: fa38d33dd708494d981d7c0bd52eae83
I20260806 06:43:57.873508 28598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:43:57.889752 28598 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:43:57.890622 28598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:43:57.904588 28598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e44bf5b35e4b258573beff174871c3: Generated new TSK 0
I20260806 06:43:57.904807 28598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:43:58.795470 28565 catalog_manager.cc:1259] Loaded cluster ID: fa38d33dd708494d981d7c0bd52eae83
I20260806 06:43:58.795559 28565 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: loading cluster ID for follower catalog manager: success
I20260806 06:43:58.796319 28565 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: acquiring CA information for follower catalog manager: success
I20260806 06:43:58.796554 28565 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8a1b9dc7db8a49388ac58355c626aca1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:43:58.815219 28580 catalog_manager.cc:1259] Loaded cluster ID: fa38d33dd708494d981d7c0bd52eae83
I20260806 06:43:58.815295 28580 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: loading cluster ID for follower catalog manager: success
I20260806 06:43:58.816262 28580 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: acquiring CA information for follower catalog manager: success
I20260806 06:43:58.816597 28580 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a974474493f4821bbae6552df1fa75a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:43:58.966434 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28311
I20260806 06:43:58.972757 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28395
I20260806 06:43:58.977803 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28470
2026-08-06T06:43:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2801 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:43:59 dist-test-slave-1xrh krb5kdc[28616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:43:59 dist-test-slave-1xrh krb5kdc[28616](info): set up 2 sockets
Aug 06 06:43:59 dist-test-slave-1xrh krb5kdc[28616](info): commencing operation
krb5kdc: starting...
W20260806 06:44:00.054656 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:44:00.175632 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35899
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:35899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:00.253612 28631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:00.253822 28631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:00.253844 28631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:00.254988 28631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:00.255031 28631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:00.255052 28631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:00.255105 28631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:00.255158 28631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:00.256625 28631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:35899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:00.256872 28631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:00.257117 28631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:00.261963 28644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:00.261917 28636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:00.261907 28642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:00.261945 28631 server_base.cc:1034] running on GCE node
I20260806 06:44:00.262310 28631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:00.262531 28631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:00.263674 28631 hybrid_clock.cc:648] HybridClock initialized: now 1785998640263649 us; error 41 us; skew 500 ppm
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998640, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:00.270375 28631 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:00.270860 28631 webserver.cc:460] Webserver started at http://127.14.51.126:38419/ using document root <none> and password file <none>
I20260806 06:44:00.271088 28631 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:00.271179 28631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:00.271354 28631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:00.272387 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3"
format_stamp: "Formatted at 2026-08-06 06:44:00 on dist-test-slave-1xrh"
server_key: "bc91dbda5a7157452ab2ef7af745551b"
server_key_iv: "7302745ccc25383254bdae6ecad28a75"
server_key_version: "encryptionkey@0"
I20260806 06:44:00.272739 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3"
format_stamp: "Formatted at 2026-08-06 06:44:00 on dist-test-slave-1xrh"
server_key: "bc91dbda5a7157452ab2ef7af745551b"
server_key_iv: "7302745ccc25383254bdae6ecad28a75"
server_key_version: "encryptionkey@0"
I20260806 06:44:00.274112 28631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:44:00.274845 28651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:00.275023 28631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:00.275123 28631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3"
format_stamp: "Formatted at 2026-08-06 06:44:00 on dist-test-slave-1xrh"
server_key: "bc91dbda5a7157452ab2ef7af745551b"
server_key_iv: "7302745ccc25383254bdae6ecad28a75"
server_key_version: "encryptionkey@0"
I20260806 06:44:00.275231 28631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:00.288335 28631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:00.290419 28631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:00.290570 28631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:00.294695 28631 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:35899
I20260806 06:44:00.294765 28704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:35899 every 8 connection(s)
I20260806 06:44:00.295171 28631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:00.296097 28705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:00.298463 28705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3: Bootstrap starting.
I20260806 06:44:00.299022 28705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:00.299345 28705 log.cc:824] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:00.299821 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28631
I20260806 06:44:00.299934 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:44:00.300137 14541 external_mini_cluster.cc:1428] Setting key 96bbf1f0705b7d6f0098c550dd6f7f31
I20260806 06:44:00.300145 28705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3: No bootstrap required, opened a new log
I20260806 06:44:00.302492 28705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } }
I20260806 06:44:00.302623 28705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:00.302644 28705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70cf5f72aa784f86a6a441eb0ca1a1e3, State: Initialized, Role: FOLLOWER
I20260806 06:44:00.302767 28705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } }
I20260806 06:44:00.302841 28705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:00.302863 28705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:00.302929 28705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:00.303529 28705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } }
I20260806 06:44:00.303614 28705 leader_election.cc:302] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70cf5f72aa784f86a6a441eb0ca1a1e3; no voters:
I20260806 06:44:00.303797 28705 leader_election.cc:288] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:44:00.303834 28710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:00.303952 28710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [term 1 LEADER]: Becoming Leader. State: Replica: 70cf5f72aa784f86a6a441eb0ca1a1e3, State: Running, Role: LEADER
I20260806 06:44:00.304062 28705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:00.304039 28710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } }
I20260806 06:44:00.304364 28713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70cf5f72aa784f86a6a441eb0ca1a1e3. Latest consensus state: current_term: 1 leader_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } } }
I20260806 06:44:00.304375 28712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf5f72aa784f86a6a441eb0ca1a1e3" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35899 } } }
I20260806 06:44:00.304457 28713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:00.304486 28712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:00.304865 28716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:00.305445 28716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:44:00.307456 28716 catalog_manager.cc:1347] Generated new cluster ID: ff1bb3d9904a46edb50bc22f8d4fa7aa
I20260806 06:44:00.307512 28716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:00.314368 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:00.300632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.14.51.126:35899)
0806 06:44:00.300809 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:00.300817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:00.300848 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:44:00.301451 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:00.301458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:00.301503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:00.301615 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:00.302451 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:00.303983 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:00.305136 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:00.305735 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:00.305838 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:00.308672 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:00.308676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:00.308676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:00.308688 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:00.312087 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:00.312535 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:00.312538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:00.312539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:00.312672 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:00.312900 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:00.312901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:00.312901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:00.313220 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:00.313386 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:00.313935 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 06:44:00.314099 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260806 06:44:00.318521 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:44:00.320739 28716 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:44:00.321460 28716 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:44:00.340404 28716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70cf5f72aa784f86a6a441eb0ca1a1e3: Generated new TSK 0
I20260806 06:44:00.340608 28716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:44:00.372542 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28631
2026-08-06T06:44:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1395 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28734](info): set up 2 sockets
Aug 06 06:44:00 dist-test-slave-1xrh krb5kdc[28734](info): commencing operation
krb5kdc: starting...
W20260806 06:44:01.421841 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:44:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:44:01.542022 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:01.622452 28747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:01.622668 28747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:01.622705 28747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:01.623894 28747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:01.623958 28747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:01.623982 28747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:01.624011 28747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:01.624040 28747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:01.625509 28747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:01.625788 28747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:01.626003 28747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:01.630633 28752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:01.630621 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:01.630625 28753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:01.630937 28747 server_base.cc:1034] running on GCE node
I20260806 06:44:01.631222 28747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:01.631484 28747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:01.632627 28747 hybrid_clock.cc:648] HybridClock initialized: now 1785998641632615 us; error 27 us; skew 500 ppm
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:01.635097 28747 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:01.635545 28747 webserver.cc:460] Webserver started at http://127.14.51.126:45623/ using document root <none> and password file <none>
I20260806 06:44:01.635761 28747 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:01.635834 28747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:01.635983 28747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:01.636971 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "372f9397825c4ebfb0bc2f31e9923935"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "92d0f70a9cb928764deec5758118e16f"
server_key_iv: "8622db1b04e5291e55bdbba77b259c43"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.637322 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "372f9397825c4ebfb0bc2f31e9923935"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "92d0f70a9cb928764deec5758118e16f"
server_key_iv: "8622db1b04e5291e55bdbba77b259c43"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.638628 28747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:44:01.639426 28762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.639578 28747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:01.639673 28747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "372f9397825c4ebfb0bc2f31e9923935"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "92d0f70a9cb928764deec5758118e16f"
server_key_iv: "8622db1b04e5291e55bdbba77b259c43"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.639750 28747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:01.657683 28747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:01.659503 28747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:01.659669 28747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:01.663405 28747 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:37131
I20260806 06:44:01.663455 28814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:37131 every 8 connection(s)
I20260806 06:44:01.663787 28747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:01.664736 28815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:01.666146 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28747
I20260806 06:44:01.666256 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:44:01.666466 14541 external_mini_cluster.cc:1428] Setting key b8fadd20b693025c67c4ef5fab32cb45
I20260806 06:44:01.667332 28815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap starting.
I20260806 06:44:01.667968 28815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:01.668303 28815 log.cc:824] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:01.669032 28815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: No bootstrap required, opened a new log
I20260806 06:44:01.671211 28815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:01.671384 28815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:01.671425 28815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Initialized, Role: FOLLOWER
I20260806 06:44:01.671563 28815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:01.671661 28815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:01.671720 28815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:01.671778 28815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:01.672307 28815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:01.672420 28815 leader_election.cc:302] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 372f9397825c4ebfb0bc2f31e9923935; no voters:
I20260806 06:44:01.672612 28815 leader_election.cc:288] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:44:01.672634 28820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:01.672750 28820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 LEADER]: Becoming Leader. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Running, Role: LEADER
I20260806 06:44:01.672830 28820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:01.672869 28815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:01.673171 28820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:01.673312 28820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:01.673502 28820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 372f9397825c4ebfb0bc2f31e9923935. Latest consensus state: current_term: 1 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:01.673631 28820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:01.673991 28828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:01.674494 28828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:44:01.676029 28828 catalog_manager.cc:1347] Generated new cluster ID: 384e1e3f916a4974a5810731cf4cd264
I20260806 06:44:01.676086 28828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:44:01.677764 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:01.666983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.14.51.126:37131)
0806 06:44:01.667183 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:01.667190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:01.667213 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:44:01.667654 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:01.667661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:01.667724 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:01.667857 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:01.668693 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.669662 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.670763 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.671206 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.671295 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:01.674032 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:01.674036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:01.674037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:01.674048 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:01.676117 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:01.676525 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:01.676528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:01.676528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:01.676634 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:01.676842 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:01.676843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:01.676844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:01.677162 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:01.677324 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:01.677433 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:44:01.677570 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260806 06:44:01.691709 28828 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:44:01.692178 28828 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:44:01.694765 28828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Generated new TSK 0
I20260806 06:44:01.694923 28828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:44:01.725950 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:01.805764 28842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:01.805968 28842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:01.806005 28842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:01.807192 28842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:01.807246 28842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:01.807291 28842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:44:01.808712 28842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:01.808961 28842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:01.809170 28842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:01.813810 28850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:01.813823 28848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:01.813874 28842 server_base.cc:1034] running on GCE node
W20260806 06:44:01.813808 28847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:01.814224 28842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:01.814484 28842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:01.815634 28842 hybrid_clock.cc:648] HybridClock initialized: now 1785998641815629 us; error 36 us; skew 500 ppm
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:01.818102 28842 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:44:01.818483 28842 webserver.cc:460] Webserver started at http://127.14.51.65:39617/ using document root <none> and password file <none>
I20260806 06:44:01.818693 28842 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:01.818755 28842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:01.818910 28842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:01.819862 28842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "b96dbc7b852b4d0985028f9bb48ad779"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "7d2b7ae190991b1287c7f9271b9dbf01"
server_key_iv: "efe120d4f8d9673c3dab66839d216845"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.820204 28842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "b96dbc7b852b4d0985028f9bb48ad779"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "7d2b7ae190991b1287c7f9271b9dbf01"
server_key_iv: "efe120d4f8d9673c3dab66839d216845"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.821421 28842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:44:01.822137 28857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.822299 28842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:01.822387 28842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "b96dbc7b852b4d0985028f9bb48ad779"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "7d2b7ae190991b1287c7f9271b9dbf01"
server_key_iv: "efe120d4f8d9673c3dab66839d216845"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.822464 28842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:01.840215 28842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:01.842465 28842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:01.842607 28842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:01.842809 28842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:01.843096 28842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:01.843173 28842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.843212 28842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:01.843274 28842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.848932 28842 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44741
I20260806 06:44:01.848977 28971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44741 every 8 connection(s)
I20260806 06:44:01.849252 28842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:44:01.850459 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28842
I20260806 06:44:01.850569 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:44:01.850744 14541 external_mini_cluster.cc:1428] Setting key 570150cbbab33138adedd30d31b7952b
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:01.858636 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:01.849903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:39825 (local address 127.14.51.126:37131)
0806 06:44:01.849979 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:01.849981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:01.850132 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:44:01.850665 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:01.850666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:01.850679 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:01.850734 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:01.851656 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.852162 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.853079 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.853281 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.853350 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:01.855829 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:01.855831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:01.855831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:01.855837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:01.857203 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:01.857718 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:01.857718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:01.857718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:01.857764 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:01.858007 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:01.858007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:01.858007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:01.858098 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:01.858150 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:01.858504 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 06:44:01.858572 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:44:01.858892 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:01.849839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:39825)
0806 06:44:01.850015 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:01.850061 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:01.850545 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:01.850795 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:01.850799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:01.851390 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:01.851564 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:01.851568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.852252 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.852253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:01.852996 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:01.852998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:01.853361 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:01.853362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:01.853476 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:01.853989 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:01.853997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:01.855728 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:01.857288 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:01.857295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:01.857298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:01.857625 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:01.857822 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:01.857822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:01.857823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:01.857947 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:01.858166 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:01.858172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:01.858352 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 06:44:01.858493 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 06:44:01.858672 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
I20260806 06:44:01.859433 28972 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:01.859566 28972 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:01.859756 28972 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:01.860431 28779 ts_manager.cc:194] Registered new tserver with Master: b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741)
I20260806 06:44:01.861032 28779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:39825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:44:01.877722 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:01.957625 28977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:01.957823 28977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:01.957844 28977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:01.958864 28977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:01.958899 28977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:01.958923 28977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:44:01.960225 28977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:01.960388 28977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:01.960551 28977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:01.965611 28982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:01.965612 28985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:01.965612 28983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:01.965966 28977 server_base.cc:1034] running on GCE node
I20260806 06:44:01.966190 28977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:01.966437 28977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:01.967617 28977 hybrid_clock.cc:648] HybridClock initialized: now 1785998641967600 us; error 26 us; skew 500 ppm
Aug 06 06:44:01 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:01.970083 28977 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:44:01.970500 28977 webserver.cc:460] Webserver started at http://127.14.51.66:43735/ using document root <none> and password file <none>
I20260806 06:44:01.970721 28977 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:01.970782 28977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:01.970923 28977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:01.971942 28977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "15dcbf00214343dc950b01a4e85d4be9"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "2be880aec1551d5cc78cb518e4982c7b"
server_key_iv: "c3842503fa24913a99ce694349acfc56"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.972288 28977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "15dcbf00214343dc950b01a4e85d4be9"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "2be880aec1551d5cc78cb518e4982c7b"
server_key_iv: "c3842503fa24913a99ce694349acfc56"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.973646 28977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:44:01.974502 28992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.974704 28977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:01.974792 28977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "15dcbf00214343dc950b01a4e85d4be9"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "2be880aec1551d5cc78cb518e4982c7b"
server_key_iv: "c3842503fa24913a99ce694349acfc56"
server_key_version: "encryptionkey@0"
I20260806 06:44:01.974858 28977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:01.992280 28977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:01.994321 28977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:01.994459 28977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:01.994714 28977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:01.995014 28977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:01.995067 28977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:01.995106 28977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:01.995152 28977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.000783 28977 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:41359
I20260806 06:44:02.000856 29109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:41359 every 8 connection(s)
I20260806 06:44:02.001178 28977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:44:02.002168 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 28977
I20260806 06:44:02.002276 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:44:02.002450 14541 external_mini_cluster.cc:1428] Setting key 01c2aa84eb7f3776eda69f32ceb20651
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998641, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:44:02.012144 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.001902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:38909 (local address 127.14.51.126:37131)
0806 06:44:02.001986 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.001989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.002181 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:44:02.003033 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.003034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.003043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.003107 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.004495 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.005193 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.006177 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.006515 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.006591 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.009160 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.009162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.009162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.009169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.010650 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.011173 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.011174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.011174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.011229 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.011501 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.011502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.011502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.011609 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.011657 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.012003 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.012079 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:44:02.012352 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.001871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:38909)
0806 06:44:02.002064 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.002099 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.002721 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.003260 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.003265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.004128 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.004397 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.004403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.005285 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.005286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.006093 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.006095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.006601 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.006602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.006699 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.007240 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.007251 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.009067 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.010735 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.010741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.010743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.011063 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.011295 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.011296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.011296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.011435 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.011670 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.011677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.011857 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.011971 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.012116 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":80,"threads_started":1}
I20260806 06:44:02.012873 29110 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:02.012955 29110 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:02.013103 29110 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:02.013422 28779 ts_manager.cc:194] Registered new tserver with Master: 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359)
I20260806 06:44:02.013837 28779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:38909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:44:02.028170 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:02.105969 29115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:02.106202 29115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:02.106241 29115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:02.107436 29115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:02.107494 29115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:02.107537 29115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:44:02.108974 29115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:02.109233 29115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:02.109453 29115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:44:02.114423 29115 server_base.cc:1034] running on GCE node
W20260806 06:44:02.114375 29120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.114311 29121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.114311 29123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.114775 29115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:02.114962 29115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:02.116092 29115 hybrid_clock.cc:648] HybridClock initialized: now 1785998642116071 us; error 37 us; skew 500 ppm
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:02.118472 29115 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:44:02.118872 29115 webserver.cc:460] Webserver started at http://127.14.51.67:39761/ using document root <none> and password file <none>
I20260806 06:44:02.119092 29115 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:02.119179 29115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:02.119323 29115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:02.120327 29115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
format_stamp: "Formatted at 2026-08-06 06:44:02 on dist-test-slave-1xrh"
server_key: "fc83682e9390fd6b94e67b78e447a0d5"
server_key_iv: "073032af5736b9669a2ded589b2f71b3"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.120676 29115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
format_stamp: "Formatted at 2026-08-06 06:44:02 on dist-test-slave-1xrh"
server_key: "fc83682e9390fd6b94e67b78e447a0d5"
server_key_iv: "073032af5736b9669a2ded589b2f71b3"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.121901 29115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:44:02.122653 29130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.122812 29115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:02.122903 29115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
format_stamp: "Formatted at 2026-08-06 06:44:02 on dist-test-slave-1xrh"
server_key: "fc83682e9390fd6b94e67b78e447a0d5"
server_key_iv: "073032af5736b9669a2ded589b2f71b3"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.122978 29115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:02.151196 29115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:02.152808 29115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:02.152953 29115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:02.153160 29115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:02.153445 29115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:02.153493 29115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.153533 29115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:02.153622 29115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.159232 29115 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:42705
I20260806 06:44:02.159277 29245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:42705 every 8 connection(s)
I20260806 06:44:02.159587 29115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:44:02.162684 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29115
I20260806 06:44:02.162770 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:44:02.162950 14541 external_mini_cluster.cc:1428] Setting key d6a94204b9bad741becc5152ce6d8aff
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:02.168402 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.160228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:47271 (local address 127.14.51.126:37131)
0806 06:44:02.160270 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.160273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.160481 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:44:02.160969 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.160970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.160980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.161026 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.161818 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.162300 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.163466 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.163705 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.163778 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.165833 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.165835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.165835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.165843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.167117 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.167544 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.167545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.167545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.167598 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.167899 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.167900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.167900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.168014 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.168081 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.168280 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.168342 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:44:02.168574 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.160182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:47271)
0806 06:44:02.160319 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.160384 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.160888 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.161083 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.161086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.161578 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.161744 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.161747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.162371 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.162372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.163396 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.163398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.163795 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.163821 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.164013 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.164433 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.164440 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.165751 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.167247 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.167253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.167254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.167477 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.167655 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.167655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.167656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.167752 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.168078 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.168082 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.168179 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.168253 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.168365 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260806 06:44:02.169020 29246 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:02.169104 29246 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:02.169261 29246 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:02.169600 28779 ts_manager.cc:194] Registered new tserver with Master: bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
I20260806 06:44:02.169934 28779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:47271
I20260806 06:44:02.174710 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:44:02.181681 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.176209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.14.51.126:37131)
0806 06:44:02.176271 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.176273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.176425 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:44:02.176465 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.176466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.176473 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.176518 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.177036 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.177558 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.178243 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.178412 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.178461 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.179694 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.179695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.179695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.179701 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.180778 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.181065 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.181065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.181066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.181110 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.181258 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.181258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.181259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.181373 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.181443 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.181519 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.181579 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:44:02.183257 28779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:44:02.184755 28779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:44:02.190609 29257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.186755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:42705 (local address 127.0.0.1:49788)
0806 06:44:02.187051 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.187061 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.187115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.187403 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.187406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.187408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.187488 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.187490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.188359 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.188361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.189383 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.189386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.190301 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.190302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.190407 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.190420 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.190494 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.190539 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":59,"thread_start_us":52,"threads_started":1}
I20260806 06:44:02.190649 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.186832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.14.51.67:42705)
0806 06:44:02.187231 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.187235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.187243 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:44:02.187274 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.187276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.187289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.187491 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.187551 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.188264 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.189480 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.190214 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.190291 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.190358 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.190523 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.190573 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":82,"threads_started":1}
I20260806 06:44:02.190883 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.186748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:41359 (local address 127.0.0.1:59836)
0806 06:44:02.186900 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.186938 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.187004 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.187403 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.187406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.187411 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.187488 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.187491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.188453 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.188454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.189661 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.189663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.190641 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.190642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.190716 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.190720 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.190777 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.190817 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260806 06:44:02.190905 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.187011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.14.51.66:41359)
0806 06:44:02.187203 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.187208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.187219 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:44:02.187261 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.187263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.187280 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.187372 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.187584 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.188343 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.189745 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.190446 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.190559 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.190614 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.190793 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.190832 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260806 06:44:02.191417 29179 tablet_service.cc:1467] Processing CreateTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f (DEFAULT_TABLE table=test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.191717 29179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f70d4168e4c41c1a995c88c56b8993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.191529 29042 tablet_service.cc:1467] Processing CreateTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f (DEFAULT_TABLE table=test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.191920 29042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f70d4168e4c41c1a995c88c56b8993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.194017 29260 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Bootstrap starting.
I20260806 06:44:02.194386 29261 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Bootstrap starting.
I20260806 06:44:02.195046 29260 tablet_bootstrap.cc:654] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.195156 29261 tablet_bootstrap.cc:654] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.195585 29260 log.cc:824] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.195748 29261 log.cc:824] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.196026 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.186943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44741 (local address 127.0.0.1:46404)
0806 06:44:02.188384 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.188393 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.188458 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.188671 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.188672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.188674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.188728 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.188730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.189896 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.189898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.194494 (+ 4596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.194497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.195725 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.195726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.195807 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.195858 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.195920 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.195961 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":51,"threads_started":1}
I20260806 06:44:02.196069 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.187060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.14.51.65:44741)
0806 06:44:02.187260 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.187264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.188521 (+ 1257us) server_negotiation.cc:407] Connection header received
0806 06:44:02.188564 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.188566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.188582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.188656 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.188805 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.189809 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.194629 (+ 4820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.195577 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.195662 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.195720 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.195949 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.195990 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":110,"threads_started":1}
I20260806 06:44:02.196326 29260 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: No bootstrap required, opened a new log
I20260806 06:44:02.196420 29260 ts_tablet_manager.cc:1397] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:44:02.196676 29261 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: No bootstrap required, opened a new log
I20260806 06:44:02.196753 29261 ts_tablet_manager.cc:1397] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.196801 28905 tablet_service.cc:1467] Processing CreateTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f (DEFAULT_TABLE table=test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.197127 28905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f70d4168e4c41c1a995c88c56b8993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.198230 29261 raft_consensus.cc:348] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.198381 29261 raft_consensus.cc:374] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.198424 29261 raft_consensus.cc:729] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15dcbf00214343dc950b01a4e85d4be9, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.198339 29260 raft_consensus.cc:348] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.198496 29260 raft_consensus.cc:374] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.198551 29260 raft_consensus.cc:729] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1c9b5a160a4116b54d7f9277b21f36, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.198570 29261 consensus_queue.cc:260] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.198661 29260 consensus_queue.cc:260] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.198846 29261 ts_tablet_manager.cc:1428] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.198916 29260 ts_tablet_manager.cc:1428] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:44:02.199038 29246 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
I20260806 06:44:02.200183 29110 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
I20260806 06:44:02.200215 29266 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap starting.
I20260806 06:44:02.200835 29266 tablet_bootstrap.cc:654] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.201282 29266 log.cc:824] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.201934 29266 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: No bootstrap required, opened a new log
I20260806 06:44:02.202013 29266 ts_tablet_manager.cc:1397] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:44:02.203553 29266 raft_consensus.cc:348] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.203684 29266 raft_consensus.cc:374] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.203727 29266 raft_consensus.cc:729] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96dbc7b852b4d0985028f9bb48ad779, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.203874 29266 consensus_queue.cc:260] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.204146 28972 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
I20260806 06:44:02.204155 29266 ts_tablet_manager.cc:1428] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.214227 29268 raft_consensus.cc:482] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:02.214349 29268 raft_consensus.cc:504] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.214645 29268 leader_election.cc:288] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
I20260806 06:44:02.217226 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.214786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:41359 (local address 127.14.51.65:59283)
0806 06:44:02.214839 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.214850 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.214899 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.215041 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.215053 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.215055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.215109 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.215110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.215715 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.215716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.216421 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.216423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.216970 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.216971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.217060 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.217063 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.217125 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.217163 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:44:02.217221 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.214897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:59283 (local address 127.14.51.66:41359)
0806 06:44:02.214938 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.214940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.214947 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:44:02.214970 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.214971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.214978 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.215021 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.215211 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.215639 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.216523 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.216892 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.216973 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.216981 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.217130 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.217162 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:44:02.217521 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.214863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:56237 (local address 127.14.51.67:42705)
0806 06:44:02.214915 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.214917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.215174 (+ 257us) server_negotiation.cc:407] Connection header received
0806 06:44:02.215232 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.215233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.215241 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.215292 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.215453 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.215898 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.216779 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.217137 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.217213 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.217220 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.217422 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.217467 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:44:02.217532 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.214786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:42705 (local address 127.14.51.65:56237)
0806 06:44:02.215091 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.215105 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.215185 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.215296 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.215297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.215298 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.215386 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.215387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.215975 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.215977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.216696 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.216698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.217228 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.217229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.217306 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.217312 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.217413 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.217460 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":140,"thread_start_us":96,"threads_started":1}
I20260806 06:44:02.217581 29062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f70d4168e4c41c1a995c88c56b8993f" candidate_uuid: "b96dbc7b852b4d0985028f9bb48ad779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15dcbf00214343dc950b01a4e85d4be9" is_pre_election: true
I20260806 06:44:02.217727 29062 raft_consensus.cc:2454] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96dbc7b852b4d0985028f9bb48ad779 in term 0.
I20260806 06:44:02.217954 28859 leader_election.cc:302] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dcbf00214343dc950b01a4e85d4be9, b96dbc7b852b4d0985028f9bb48ad779; no voters:
I20260806 06:44:02.217939 29199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f70d4168e4c41c1a995c88c56b8993f" candidate_uuid: "b96dbc7b852b4d0985028f9bb48ad779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" is_pre_election: true
I20260806 06:44:02.218091 29199 raft_consensus.cc:2454] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96dbc7b852b4d0985028f9bb48ad779 in term 0.
I20260806 06:44:02.218117 29268 raft_consensus.cc:2781] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:44:02.218189 29268 raft_consensus.cc:482] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:44:02.218236 29268 raft_consensus.cc:3037] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.218930 29268 raft_consensus.cc:504] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.219075 29268 leader_election.cc:288] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 1 election: Requested vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
I20260806 06:44:02.219244 29062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f70d4168e4c41c1a995c88c56b8993f" candidate_uuid: "b96dbc7b852b4d0985028f9bb48ad779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15dcbf00214343dc950b01a4e85d4be9"
I20260806 06:44:02.219264 29199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f70d4168e4c41c1a995c88c56b8993f" candidate_uuid: "b96dbc7b852b4d0985028f9bb48ad779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
I20260806 06:44:02.219336 29199 raft_consensus.cc:3037] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.219336 29062 raft_consensus.cc:3037] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.220144 29199 raft_consensus.cc:2454] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96dbc7b852b4d0985028f9bb48ad779 in term 1.
I20260806 06:44:02.220144 29062 raft_consensus.cc:2454] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96dbc7b852b4d0985028f9bb48ad779 in term 1.
I20260806 06:44:02.220314 28859 leader_election.cc:302] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dcbf00214343dc950b01a4e85d4be9, b96dbc7b852b4d0985028f9bb48ad779; no voters:
I20260806 06:44:02.220417 29268 raft_consensus.cc:2781] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:02.220561 29268 raft_consensus.cc:686] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 LEADER]: Becoming Leader. State: Replica: b96dbc7b852b4d0985028f9bb48ad779, State: Running, Role: LEADER
I20260806 06:44:02.220656 29268 consensus_queue.cc:237] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.221319 28779 catalog_manager.cc:5495] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 reported cstate change: term changed from 0 to 1, leader changed from <none> to b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65). New cstate: current_term: 1 leader_uuid: "b96dbc7b852b4d0985028f9bb48ad779" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } health_report { overall_health: HEALTHY } } }
I20260806 06:44:02.230058 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.227992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.14.51.65:44741)
0806 06:44:02.228027 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.228029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.228038 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:44:02.228103 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.228103 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.228135 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.228188 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:02.228289 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.228705 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.229380 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.229575 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.229633 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.229747 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:02.229838 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:44:02.229899 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.229931 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.229946 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:44:02.231550 29062 raft_consensus.cc:1261] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Refusing update from remote peer b96dbc7b852b4d0985028f9bb48ad779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:02.231545 29199 raft_consensus.cc:1261] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Refusing update from remote peer b96dbc7b852b4d0985028f9bb48ad779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:02.231880 29270 consensus_queue.cc:1035] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:02.231993 29268 consensus_queue.cc:1035] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:02.234445 29274 mvcc.cc:205] Tried to move back new op lower bound from 7315450438578053120 to 7315450438536413184. Current Snapshot: MvccSnapshot[applied={T|T < 7315450438578053120}]
I20260806 06:44:02.236939 28778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:02.237020 28778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:02.237675 28778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Sending DeleteTablet for 3 replicas of tablet 4f70d4168e4c41c1a995c88c56b8993f
I20260806 06:44:02.237998 28905 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46404
I20260806 06:44:02.238041 29179 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:49788
I20260806 06:44:02.238054 29042 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:59836
I20260806 06:44:02.238271 29285 tablet_replica.cc:330] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: stopping tablet replica
I20260806 06:44:02.238273 29283 tablet_replica.cc:330] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: stopping tablet replica
I20260806 06:44:02.238344 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28842
I20260806 06:44:02.238361 29285 raft_consensus.cc:2229] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:02.238365 29283 raft_consensus.cc:2229] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:44:02.238474 29283 raft_consensus.cc:2258] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.240533 29285 raft_consensus.cc:2258] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.240772 29284 tablet_replica.cc:330] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: stopping tablet replica
I20260806 06:44:02.240854 29284 raft_consensus.cc:2229] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:02.240939 29284 raft_consensus.cc:2258] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.241300 29284 ts_tablet_manager.cc:1905] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:02.241300 29285 ts_tablet_manager.cc:1905] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:02.242969 29284 ts_tablet_manager.cc:1918] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:02.242935 29285 ts_tablet_manager.cc:1918] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:02.243077 29285 log.cc:1196] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/4f70d4168e4c41c1a995c88c56b8993f
I20260806 06:44:02.243110 29284 log.cc:1196] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/4f70d4168e4c41c1a995c88c56b8993f
I20260806 06:44:02.243408 29284 ts_tablet_manager.cc:1939] T 4f70d4168e4c41c1a995c88c56b8993f P 15dcbf00214343dc950b01a4e85d4be9: Deleting consensus metadata
I20260806 06:44:02.243393 29285 ts_tablet_manager.cc:1939] T 4f70d4168e4c41c1a995c88c56b8993f P bb1c9b5a160a4116b54d7f9277b21f36: Deleting consensus metadata
I20260806 06:44:02.243769 28764 catalog_manager.cc:4841] TS 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): tablet 4f70d4168e4c41c1a995c88c56b8993f (table test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]) successfully deleted
I20260806 06:44:02.243783 28765 catalog_manager.cc:4841] TS bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): tablet 4f70d4168e4c41c1a995c88c56b8993f (table test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]) successfully deleted
W20260806 06:44:02.246479 28764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:44741 (error 108)
I20260806 06:44:02.246739 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28977
I20260806 06:44:02.246927 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.246776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44741 (local address 127.0.0.1:46426)
0806 06:44:02.246849 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.246892 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 06:44:02.248301 28764 catalog_manager.cc:4584] TS b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f70d4168e4c41c1a995c88c56b8993f: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44741: connect: Connection refused (error 111)
I20260806 06:44:02.254921 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29115
I20260806 06:44:02.259994 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 28747
I20260806 06:44:02.265388 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--webserver_interface=127.14.51.126
--webserver_port=45623
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:02.346127 29287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:02.346323 29287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:02.346344 29287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:02.347513 29287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:02.347553 29287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:02.347568 29287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:02.347612 29287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:02.347636 29287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:02.349100 29287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:02.349331 29287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:02.349575 29287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:02.358992 29296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.359088 29294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.359225 29287 server_base.cc:1034] running on GCE node
W20260806 06:44:02.359094 29293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.359635 29287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:02.359931 29287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:02.361107 29287 hybrid_clock.cc:648] HybridClock initialized: now 1785998642361094 us; error 29 us; skew 500 ppm
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:02.363633 29287 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:02.364147 29287 webserver.cc:460] Webserver started at http://127.14.51.126:45623/ using document root <none> and password file <none>
I20260806 06:44:02.364383 29287 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:02.364463 29287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:02.365851 29287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:44:02.366513 29303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.366678 29287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:02.366777 29287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "372f9397825c4ebfb0bc2f31e9923935"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "92d0f70a9cb928764deec5758118e16f"
server_key_iv: "8622db1b04e5291e55bdbba77b259c43"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.367195 29287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:02.391474 29287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:02.393261 29287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:02.393428 29287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:02.397569 29287 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:37131
I20260806 06:44:02.397607 29355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:37131 every 8 connection(s)
I20260806 06:44:02.398015 29287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:02.399639 29356 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:44:02.400272 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29287
I20260806 06:44:02.400413 29356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap starting.
I20260806 06:44:02.400585 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:44741
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=39617
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:02.409155 29356 log.cc:824] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.411005 29356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:02.411360 29356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap complete.
I20260806 06:44:02.413487 29356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:02.413695 29356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.413853 29356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:02.413910 29356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:02.413966 29356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:02.414033 29356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:44:02.415058 29356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:02.415200 29356 leader_election.cc:302] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 372f9397825c4ebfb0bc2f31e9923935; no voters:
I20260806 06:44:02.415377 29356 leader_election.cc:288] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:44:02.415409 29360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:44:02.415513 29360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 LEADER]: Becoming Leader. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Running, Role: LEADER
I20260806 06:44:02.415612 29360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:02.415635 29356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:02.415904 29362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 372f9397825c4ebfb0bc2f31e9923935. Latest consensus state: current_term: 2 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:02.415990 29362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:02.416141 29361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:02.416193 29361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:02.416291 29364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:02.417181 29364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:44:02.417343 29364 catalog_manager.cc:1259] Loaded cluster ID: 384e1e3f916a4974a5810731cf4cd264
I20260806 06:44:02.417603 29364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:44:02.417882 29364 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 06:44:02.420536 29364 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 06:44:02.425735 29364 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:44:02.425926 29364 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Loaded TSK: 0
I20260806 06:44:02.426138 29364 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:44:02.494832 29358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:02.495065 29358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:02.495105 29358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:02.496361 29358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:02.496425 29358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:02.496470 29358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:44:02.497910 29358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=39617
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:02.498143 29358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:02.498358 29358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:02.504041 29385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.504070 29386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.504041 29388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.504374 29358 server_base.cc:1034] running on GCE node
I20260806 06:44:02.504554 29358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:02.504792 29358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:02.505940 29358 hybrid_clock.cc:648] HybridClock initialized: now 1785998642505930 us; error 26 us; skew 500 ppm
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:02.508324 29358 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:44:02.508766 29358 webserver.cc:460] Webserver started at http://127.14.51.65:39617/ using document root <none> and password file <none>
I20260806 06:44:02.509016 29358 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:02.509073 29358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:02.510325 29358 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:02.510959 29395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.511145 29358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:44:02.511242 29358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "b96dbc7b852b4d0985028f9bb48ad779"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "7d2b7ae190991b1287c7f9271b9dbf01"
server_key_iv: "efe120d4f8d9673c3dab66839d216845"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.511545 29358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:02.534341 29358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:02.536531 29358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:02.536667 29358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:02.536911 29358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:02.537388 29402 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:44:02.538699 29358 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:44:02.538749 29358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:02.538780 29358 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:44:02.539494 29358 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:44:02.539538 29358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:02.539574 29402 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap starting.
I20260806 06:44:02.547199 29358 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44741
I20260806 06:44:02.547278 29513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44741 every 8 connection(s)
I20260806 06:44:02.547797 29358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:44:02.550945 29402 log.cc:824] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.553225 29402 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:02.553516 29402 tablet_bootstrap.cc:492] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap complete.
I20260806 06:44:02.554672 29402 ts_tablet_manager.cc:1397] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.010s
I20260806 06:44:02.555997 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29358
I20260806 06:44:02.555846 29402 raft_consensus.cc:348] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.556165 29402 raft_consensus.cc:729] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96dbc7b852b4d0985028f9bb48ad779, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.556290 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:41359
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=43735
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:02.556337 29402 consensus_queue.cc:260] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:02.556600 29402 ts_tablet_manager.cc:1428] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:44:02.562124 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.548562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:42547)
0806 06:44:02.548822 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.548859 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.549441 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.549977 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.549981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.550333 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.550597 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.550604 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.552203 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.552204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.553486 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.553489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.554371 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.554372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.554496 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.555084 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.555093 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.558247 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.560566 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.560572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.560574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.560790 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.561071 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.561072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.561072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.561180 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.561687 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.561691 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.561791 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.561880 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.561995 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260806 06:44:02.562342 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.548680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:42547 (local address 127.14.51.126:37131)
0806 06:44:02.548904 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.548912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.548953 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:44:02.549689 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.549698 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.549760 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.549930 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.550768 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.552091 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.553608 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.554261 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.554366 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.558365 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.558369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.558369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.558383 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.560463 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.560869 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.560872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.560872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.560996 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.561242 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.561243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.561243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.561593 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.561712 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.561966 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.562113 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260806 06:44:02.562988 29514 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:02.563072 29514 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:02.563266 29514 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:02.564008 29320 ts_manager.cc:194] Registered new tserver with Master: b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741)
I20260806 06:44:02.565104 29320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:42547
I20260806 06:44:02.565912 29514 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:44:02.573423 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.564617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44741 (local address 127.0.0.1:46430)
0806 06:44:02.564887 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.564921 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.564992 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.565264 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.565271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.565632 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.565687 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.565690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.566777 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.566779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.567811 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.567815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.568228 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.568229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.568400 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.568831 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.568836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.570546 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.572277 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.572283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.572285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.572486 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.572709 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.572709 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.572710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.572771 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.573065 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.573072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.573118 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.573202 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.573317 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":86,"threads_started":1}
I20260806 06:44:02.573429 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.564683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.14.51.65:44741)
0806 06:44:02.564826 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.564831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.565001 (+ 170us) server_negotiation.cc:407] Connection header received
0806 06:44:02.565108 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.565110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.565129 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.565195 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.565776 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.566686 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.567894 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.568151 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.568204 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.570667 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.570673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.570674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.570680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.572154 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.572579 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.572582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.572582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.572638 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.572847 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.572847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.572848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.572980 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.573063 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.573250 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.573345 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260806 06:44:02.573931 29445 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f70d4168e4c41c1a995c88c56b8993f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46430
I20260806 06:44:02.574364 29523 tablet_replica.cc:330] stopping tablet replica
I20260806 06:44:02.574470 29523 raft_consensus.cc:2229] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:02.574533 29523 raft_consensus.cc:2258] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.574829 29523 ts_tablet_manager.cc:1905] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:02.577087 29523 ts_tablet_manager.cc:1918] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:02.577167 29523 log.cc:1196] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/4f70d4168e4c41c1a995c88c56b8993f
I20260806 06:44:02.577401 29523 ts_tablet_manager.cc:1939] T 4f70d4168e4c41c1a995c88c56b8993f P b96dbc7b852b4d0985028f9bb48ad779: Deleting consensus metadata
I20260806 06:44:02.577852 29305 catalog_manager.cc:4841] TS b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741): tablet 4f70d4168e4c41c1a995c88c56b8993f (table test-table [id=7c86a79f31ae4b14a0d9d4574caa54d8]) successfully deleted
W20260806 06:44:02.639807 29520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:02.640024 29520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:02.640061 29520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:02.641188 29520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:02.641253 29520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:02.641299 29520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:44:02.642719 29520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:41359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=43735
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:02.642979 29520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:02.643206 29520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:02.647953 29528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.647931 29531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.647921 29529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.648170 29520 server_base.cc:1034] running on GCE node
I20260806 06:44:02.648392 29520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:02.648617 29520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:02.649770 29520 hybrid_clock.cc:648] HybridClock initialized: now 1785998642649756 us; error 41 us; skew 500 ppm
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:02.652288 29520 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:44:02.652719 29520 webserver.cc:460] Webserver started at http://127.14.51.66:43735/ using document root <none> and password file <none>
I20260806 06:44:02.652951 29520 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:02.653031 29520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:02.654325 29520 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:02.654973 29538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.655223 29520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:02.655329 29520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "15dcbf00214343dc950b01a4e85d4be9"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "2be880aec1551d5cc78cb518e4982c7b"
server_key_iv: "c3842503fa24913a99ce694349acfc56"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.655773 29520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:02.665071 29520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:02.666692 29520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:02.666793 29520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:02.666963 29520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:02.667249 29520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:02.667291 29520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.667326 29520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:02.667343 29520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.672901 29520 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:41359
I20260806 06:44:02.672958 29655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:41359 every 8 connection(s)
I20260806 06:44:02.673307 29520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:02.681816 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29520
I20260806 06:44:02.682082 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.673968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:47465 (local address 127.14.51.126:37131)
0806 06:44:02.674021 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.674024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.674246 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:44:02.674682 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.674683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.674692 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.674742 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.675731 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.676313 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.677239 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.677445 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.677513 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.679558 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.679560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.679560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.679568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.680783 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.681158 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.681160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.681160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.681215 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.681448 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.681449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.681449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.681552 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.681614 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.681894 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.681973 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:44:02.682090 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:42705
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=39761
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:02.682178 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.673973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:47465)
0806 06:44:02.674136 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.674168 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.674602 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.674790 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.674793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.675456 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.675644 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.675648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.676389 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.676390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.677152 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.677154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.677543 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.677545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.677648 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.678137 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.678146 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.679463 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.680886 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.680892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.680894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.681087 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.681273 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.681274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.681274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.681381 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.681619 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.681624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.681736 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.681823 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.681968 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260806 06:44:02.683531 29656 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:02.683632 29656 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:02.683817 29656 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:02.684209 29320 ts_manager.cc:194] Registered new tserver with Master: 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359)
I20260806 06:44:02.684576 29320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:47465
W20260806 06:44:02.763331 29659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:02.763510 29659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:02.763532 29659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:02.764670 29659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:02.764707 29659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:02.764732 29659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:44:02.766155 29659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:42705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=39761
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:02.766371 29659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:02.766585 29659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:02.771358 29664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.771420 29667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:02.771581 29665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:02.771620 29659 server_base.cc:1034] running on GCE node
I20260806 06:44:02.771854 29659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:02.772039 29659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:02.773196 29659 hybrid_clock.cc:648] HybridClock initialized: now 1785998642773180 us; error 28 us; skew 500 ppm
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:02.775646 29659 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:44:02.776051 29659 webserver.cc:460] Webserver started at http://127.14.51.67:39761/ using document root <none> and password file <none>
I20260806 06:44:02.776217 29659 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:02.776307 29659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:02.777556 29659 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:02.778120 29674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.778275 29659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.778347 29659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
format_stamp: "Formatted at 2026-08-06 06:44:02 on dist-test-slave-1xrh"
server_key: "fc83682e9390fd6b94e67b78e447a0d5"
server_key_iv: "073032af5736b9669a2ded589b2f71b3"
server_key_version: "encryptionkey@0"
I20260806 06:44:02.778620 29659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:02.823828 29659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:02.826171 29659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:02.826289 29659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:02.826478 29659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:02.826809 29659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:02.826845 29659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.826905 29659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:02.826938 29659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:02.832499 29659 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:42705
I20260806 06:44:02.832547 29789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:42705 every 8 connection(s)
I20260806 06:44:02.832906 29659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:44:02.837775 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29659
Aug 06 06:44:02 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:02.842422 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.833634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:58823 (local address 127.14.51.126:37131)
0806 06:44:02.833725 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.833727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.833856 (+ 129us) server_negotiation.cc:407] Connection header received
0806 06:44:02.834330 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.834332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.834342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.834409 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.835457 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.835956 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.836936 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.837114 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.837163 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.839422 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.839423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.839423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.839431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.840889 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.841449 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.841450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.841450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.841517 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.841845 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.841845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.841846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.841947 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.842000 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.842290 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.842359 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:44:02.842669 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.833562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:58823)
0806 06:44:02.833733 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.833767 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.834253 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.834440 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.834461 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.835057 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:02.835334 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.835340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.836040 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.836040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.836858 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.836861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.837185 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.837186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.837284 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.837798 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:02.837841 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:02.839342 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:02.841010 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.841016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.841018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.841374 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.841593 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:02.841594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:02.841594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:02.841741 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:02.842015 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:02.842021 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:02.842162 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.842244 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.842417 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260806 06:44:02.843103 29790 heartbeater.cc:347] Connected to a master server at 127.14.51.126:37131
I20260806 06:44:02.843226 29790 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:02.843429 29790 heartbeater.cc:511] Master 127.14.51.126:37131 requested a full tablet report, sending...
I20260806 06:44:02.843853 29320 ts_manager.cc:194] Registered new tserver with Master: bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
I20260806 06:44:02.844349 29320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:58823
I20260806 06:44:02.845172 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.838174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.14.51.126:37131)
0806 06:44:02.838413 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.838415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.838420 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:44:02.838436 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.838437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.838445 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.838513 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:02.839077 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.839720 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.840427 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.840626 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.840718 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.842075 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:02.842076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:02.842076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:02.842081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:02.843855 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.844204 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.844204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.844205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.844254 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:02.844645 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:02.844646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:02.844646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:02.844766 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:02.844884 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.845037 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.845116 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
I20260806 06:44:02.845892 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:44:02.848426 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.846184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.14.51.126:37131)
0806 06:44:02.846239 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.846241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.846352 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:44:02.846437 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.846437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.846471 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.846554 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:02.846641 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.847076 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.847812 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.847988 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.848037 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.848160 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:02.848240 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:44:02.848297 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.848336 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.848353 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:44:02.848665 29320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:44:02.850178 29320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:44:02.852432 29445 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 (DEFAULT_TABLE table=test-table [id=17b36f69838d4bd0aaa97a99a6d6cdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.852630 29445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4261028c3247a69d9460e0b0db5f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.853973 29402 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap starting.
I20260806 06:44:02.854302 29402 tablet_bootstrap.cc:654] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.856129 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.852320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.14.51.67:42705)
0806 06:44:02.852492 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.852497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.852507 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:44:02.852555 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.852557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.852573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.852666 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.852787 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.853879 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855336 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.855770 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855862 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.855923 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.856015 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.856058 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260806 06:44:02.856149 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.852132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:42705 (local address 127.0.0.1:49804)
0806 06:44:02.852377 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.852386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.852453 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.852683 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.852684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.852686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.852725 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.852727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.853988 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.853989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.855261 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.855263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.855853 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.855921 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.855941 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.856001 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.856055 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260806 06:44:02.856374 29402 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: No bootstrap required, opened a new log
I20260806 06:44:02.856369 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.852198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.14.51.66:41359)
0806 06:44:02.852569 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.852573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.852582 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:44:02.852632 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.852634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.852649 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.852929 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.853077 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.854084 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855361 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.855855 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855924 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.855978 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.856193 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.856243 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":82,"threads_started":1}
I20260806 06:44:02.856442 29402 ts_tablet_manager.cc:1397] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:44:02.856760 29402 raft_consensus.cc:348] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.856838 29402 raft_consensus.cc:374] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.856894 29402 raft_consensus.cc:729] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96dbc7b852b4d0985028f9bb48ad779, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.856952 29402 consensus_queue.cc:260] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.856949 29722 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 (DEFAULT_TABLE table=test-table [id=17b36f69838d4bd0aaa97a99a6d6cdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.857081 29402 ts_tablet_manager.cc:1428] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:02.857296 29722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4261028c3247a69d9460e0b0db5f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.859372 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.852073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:41359 (local address 127.0.0.1:59846)
0806 06:44:02.852117 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.852129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.852165 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.852746 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.852747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.852749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.852793 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.852794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.854189 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.854189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.855260 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.855263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.855939 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.855940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.856024 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.856028 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.856081 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.856123 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":8}
I20260806 06:44:02.859974 29587 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 (DEFAULT_TABLE table=test-table [id=17b36f69838d4bd0aaa97a99a6d6cdc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:02.860180 29800 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Bootstrap starting.
I20260806 06:44:02.860284 29587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4261028c3247a69d9460e0b0db5f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:02.860797 29800 tablet_bootstrap.cc:654] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.861310 29800 log.cc:824] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.862063 29800 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: No bootstrap required, opened a new log
I20260806 06:44:02.862178 29800 ts_tablet_manager.cc:1397] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.863692 29802 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Bootstrap starting.
I20260806 06:44:02.864142 29800 raft_consensus.cc:348] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.864264 29800 raft_consensus.cc:374] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.864249 29802 tablet_bootstrap.cc:654] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:02.864285 29800 raft_consensus.cc:729] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1c9b5a160a4116b54d7f9277b21f36, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.864403 29800 consensus_queue.cc:260] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.864663 29800 ts_tablet_manager.cc:1428] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.864687 29790 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
I20260806 06:44:02.864846 29802 log.cc:824] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:02.865577 29802 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: No bootstrap required, opened a new log
I20260806 06:44:02.865692 29802 ts_tablet_manager.cc:1397] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.866959 29802 raft_consensus.cc:348] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.867076 29802 raft_consensus.cc:374] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:02.867098 29802 raft_consensus.cc:729] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15dcbf00214343dc950b01a4e85d4be9, State: Initialized, Role: FOLLOWER
I20260806 06:44:02.867234 29802 consensus_queue.cc:260] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.867451 29802 ts_tablet_manager.cc:1428] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:02.867492 29656 heartbeater.cc:503] Master 127.14.51.126:37131 was elected leader, sending a full tablet report...
I20260806 06:44:02.920507 29805 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:02.920603 29805 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.920931 29805 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705), b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741)
W20260806 06:44:02.923789 29657 tablet.cc:2369] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:44:02.924712 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.921166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:33361 (local address 127.14.51.67:42705)
0806 06:44:02.921208 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.921210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.921223 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:44:02.921252 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.921253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.921267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.921336 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.921483 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.922234 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.923707 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.924331 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.924420 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.924429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.924603 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.924643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:44:02.924712 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.921166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:43389 (local address 127.14.51.65:44741)
0806 06:44:02.921216 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.921218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.921352 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:44:02.921382 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.921383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.921408 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.921480 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:02.921612 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.922346 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.923707 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.924360 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.924420 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.924487 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.924603 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.924644 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:44:02.924729 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.921088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:42705 (local address 127.14.51.66:33361)
0806 06:44:02.921142 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.921155 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.921199 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.921337 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.921338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.921340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.921410 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.921412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.922324 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.922325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.923610 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.923614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.924432 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.924434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.924520 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.924525 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.924589 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.924632 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":28}
I20260806 06:44:02.924736 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.921088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44741 (local address 127.14.51.66:43389)
0806 06:44:02.921259 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.921267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:02.921336 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:02.921483 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:02.921483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:02.921485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:02.921545 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.921546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.922443 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.922444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.923610 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:02.923614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.924443 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.924444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:02.924520 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.924525 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:02.924584 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:44:02.924632 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":27,"thread_start_us":92,"threads_started":1}
I20260806 06:44:02.925236 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4261028c3247a69d9460e0b0db5f03" candidate_uuid: "15dcbf00214343dc950b01a4e85d4be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96dbc7b852b4d0985028f9bb48ad779" is_pre_election: true
I20260806 06:44:02.925266 29742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4261028c3247a69d9460e0b0db5f03" candidate_uuid: "15dcbf00214343dc950b01a4e85d4be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" is_pre_election: true
I20260806 06:44:02.925359 29466 raft_consensus.cc:2454] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15dcbf00214343dc950b01a4e85d4be9 in term 0.
I20260806 06:44:02.925418 29742 raft_consensus.cc:2454] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15dcbf00214343dc950b01a4e85d4be9 in term 0.
I20260806 06:44:02.925608 29540 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dcbf00214343dc950b01a4e85d4be9, b96dbc7b852b4d0985028f9bb48ad779; no voters:
I20260806 06:44:02.925769 29805 raft_consensus.cc:2781] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:44:02.925825 29805 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:44:02.925849 29805 raft_consensus.cc:3037] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.926592 29805 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.926723 29805 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [CANDIDATE]: Term 1 election: Requested vote from peers bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705), b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741)
I20260806 06:44:02.926865 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4261028c3247a69d9460e0b0db5f03" candidate_uuid: "15dcbf00214343dc950b01a4e85d4be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96dbc7b852b4d0985028f9bb48ad779"
I20260806 06:44:02.926872 29742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4261028c3247a69d9460e0b0db5f03" candidate_uuid: "15dcbf00214343dc950b01a4e85d4be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
I20260806 06:44:02.926965 29466 raft_consensus.cc:3037] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.926962 29742 raft_consensus.cc:3037] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:02.927713 29742 raft_consensus.cc:2454] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15dcbf00214343dc950b01a4e85d4be9 in term 1.
I20260806 06:44:02.927723 29466 raft_consensus.cc:2454] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15dcbf00214343dc950b01a4e85d4be9 in term 1.
I20260806 06:44:02.927882 29541 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15dcbf00214343dc950b01a4e85d4be9, bb1c9b5a160a4116b54d7f9277b21f36; no voters:
I20260806 06:44:02.927965 29805 raft_consensus.cc:2781] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:02.928089 29805 raft_consensus.cc:686] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 1 LEADER]: Becoming Leader. State: Replica: 15dcbf00214343dc950b01a4e85d4be9, State: Running, Role: LEADER
I20260806 06:44:02.928181 29805 consensus_queue.cc:237] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:02.928989 29320 catalog_manager.cc:5495] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "15dcbf00214343dc950b01a4e85d4be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } health_report { overall_health: UNKNOWN } } }
I20260806 06:44:02.949112 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.946773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.14.51.66:41359)
0806 06:44:02.946858 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:02.946860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:02.946869 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:44:02.946902 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:02.946903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:02.946916 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:02.946989 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:02.947091 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.947578 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.948310 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:02.948507 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:02.948579 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:02.948761 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:02.948854 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:44:02.948940 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:02.948977 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:44:02.948996 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:44:02.950667 29742 raft_consensus.cc:1261] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Refusing update from remote peer 15dcbf00214343dc950b01a4e85d4be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:02.950699 29466 raft_consensus.cc:1261] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Refusing update from remote peer 15dcbf00214343dc950b01a4e85d4be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:02.951014 29807 consensus_queue.cc:1035] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:02.951149 29805 consensus_queue.cc:1035] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:02.955437 29319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:02.955533 29319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:02.956225 29319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Sending DeleteTablet for 3 replicas of tablet 4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:02.956549 29445 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46430
I20260806 06:44:02.956566 29587 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:59846
I20260806 06:44:02.956655 29523 tablet_replica.cc:330] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: stopping tablet replica
I20260806 06:44:02.956673 29722 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4261028c3247a69d9460e0b0db5f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:02 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:49804
I20260806 06:44:02.956719 29523 raft_consensus.cc:2229] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:02.956753 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29358
I20260806 06:44:02.956774 29523 raft_consensus.cc:2258] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.956883 29820 tablet_replica.cc:330] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: stopping tablet replica
I20260806 06:44:02.956892 29821 tablet_replica.cc:330] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: stopping tablet replica
I20260806 06:44:02.956971 29821 raft_consensus.cc:2229] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:02.956970 29820 raft_consensus.cc:2229] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:44:02.957053 29821 raft_consensus.cc:2258] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.957053 29820 raft_consensus.cc:2258] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:02.957865 29820 ts_tablet_manager.cc:1905] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:02.959515 29820 ts_tablet_manager.cc:1918] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:02.959591 29820 log.cc:1196] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:02.959708 29821 ts_tablet_manager.cc:1905] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:02.959874 29820 ts_tablet_manager.cc:1939] T 4f4261028c3247a69d9460e0b0db5f03 P 15dcbf00214343dc950b01a4e85d4be9: Deleting consensus metadata
I20260806 06:44:02.960383 29305 catalog_manager.cc:4841] TS 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): tablet 4f4261028c3247a69d9460e0b0db5f03 (table test-table [id=17b36f69838d4bd0aaa97a99a6d6cdc3]) successfully deleted
I20260806 06:44:02.961658 29821 ts_tablet_manager.cc:1918] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:02.961781 29821 log.cc:1196] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:02.962078 29821 ts_tablet_manager.cc:1939] T 4f4261028c3247a69d9460e0b0db5f03 P bb1c9b5a160a4116b54d7f9277b21f36: Deleting consensus metadata
I20260806 06:44:02.962478 29306 catalog_manager.cc:4841] TS bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): tablet 4f4261028c3247a69d9460e0b0db5f03 (table test-table [id=17b36f69838d4bd0aaa97a99a6d6cdc3]) successfully deleted
W20260806 06:44:02.965266 29305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:44741 (error 108)
I20260806 06:44:02.965551 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29520
I20260806 06:44:02.965721 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:02.965602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:44741 (local address 127.0.0.1:46434)
0806 06:44:02.965646 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:02.965688 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:44:02.966334 29305 catalog_manager.cc:4584] TS b96dbc7b852b4d0985028f9bb48ad779 (127.14.51.65:44741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f4261028c3247a69d9460e0b0db5f03: Network error: Client connection negotiation failed: client connection to 127.14.51.65:44741: connect: Connection refused (error 111)
I20260806 06:44:02.971190 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29659
I20260806 06:44:02.975849 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29287
I20260806 06:44:02.981777 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--webserver_interface=127.14.51.126
--webserver_port=45623
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:03.061242 29823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:03.061431 29823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:03.061455 29823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:03.062558 29823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:03.062599 29823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:03.062613 29823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:03.062659 29823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:03.062681 29823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:03.064134 29823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:03.064381 29823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:03.064601 29823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:03.069453 29828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.069486 29831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.069483 29829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:03.069515 29823 server_base.cc:1034] running on GCE node
I20260806 06:44:03.069873 29823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:03.070165 29823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:03.071314 29823 hybrid_clock.cc:648] HybridClock initialized: now 1785998643071313 us; error 36 us; skew 500 ppm
Aug 06 06:44:03 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:03.073642 29823 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:03.074064 29823 webserver.cc:460] Webserver started at http://127.14.51.126:45623/ using document root <none> and password file <none>
I20260806 06:44:03.074280 29823 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:03.074342 29823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:03.075717 29823 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:03.076432 29838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.076602 29823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:44:03.076681 29823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "372f9397825c4ebfb0bc2f31e9923935"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "92d0f70a9cb928764deec5758118e16f"
server_key_iv: "8622db1b04e5291e55bdbba77b259c43"
server_key_version: "encryptionkey@0"
I20260806 06:44:03.076959 29823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:03.090160 29823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:03.092281 29823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:03.092423 29823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:03.096205 29823 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:37131
I20260806 06:44:03.096247 29890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:37131 every 8 connection(s)
I20260806 06:44:03.096671 29823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:03.098157 29891 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:44:03.098701 29891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap starting.
I20260806 06:44:03.106588 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29823
I20260806 06:44:03.106935 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:44741
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=39617
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:03.107295 29891 log.cc:824] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:03.109604 29891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:03.109907 29891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935: Bootstrap complete.
I20260806 06:44:03.112041 29891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:03.112263 29891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Initialized, Role: FOLLOWER
I20260806 06:44:03.112449 29891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:03.112537 29891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:03.112587 29891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:03.112633 29891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:44:03.113530 29891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:03.113643 29891 leader_election.cc:302] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 372f9397825c4ebfb0bc2f31e9923935; no voters:
I20260806 06:44:03.113842 29891 leader_election.cc:288] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:44:03.113884 29895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:44:03.114043 29895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [term 3 LEADER]: Becoming Leader. State: Replica: 372f9397825c4ebfb0bc2f31e9923935, State: Running, Role: LEADER
I20260806 06:44:03.114112 29891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:03.114218 29895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } }
I20260806 06:44:03.114502 29897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 372f9397825c4ebfb0bc2f31e9923935. Latest consensus state: current_term: 3 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:03.114512 29896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "372f9397825c4ebfb0bc2f31e9923935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "372f9397825c4ebfb0bc2f31e9923935" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 37131 } } }
I20260806 06:44:03.114693 29897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:03.114717 29896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372f9397825c4ebfb0bc2f31e9923935 [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:03.115115 29904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:03.116072 29904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:44:03.116256 29904 catalog_manager.cc:1259] Loaded cluster ID: 384e1e3f916a4974a5810731cf4cd264
I20260806 06:44:03.116588 29904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 06:44:03.116922 29904 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb9240c89ad google::LogMessage::Fail() at ??:0
@ 0x7fb9240cb88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb9240c84de google::LogMessage::Flush() at ??:0
@ 0x7fb9240c9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb92bfe90f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb92bff28b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb925f960d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb925f8b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb926def6db start_thread at ??:0
@ 0x7fb922bc661f clone at ??:0
W20260806 06:44:03.221661 29894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:03.221940 29894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:03.221978 29894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:03.223783 29894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:03.223860 29894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:03.223904 29894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:44:03.226328 29894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=39617
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:03.226593 29894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:03.226877 29894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:03.232342 29930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.232411 29932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:03.232549 29894 server_base.cc:1034] running on GCE node
W20260806 06:44:03.232647 29929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:03.232872 29894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:03.233166 29894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:03.234369 29894 hybrid_clock.cc:648] HybridClock initialized: now 1785998643234349 us; error 34 us; skew 500 ppm
Aug 06 06:44:03 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:03.237143 29894 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:44:03.237535 29894 webserver.cc:460] Webserver started at http://127.14.51.65:39617/ using document root <none> and password file <none>
I20260806 06:44:03.237759 29894 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:03.237823 29894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:03.239250 29894 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:44:03.240078 29939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.240273 29894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:03.240363 29894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "b96dbc7b852b4d0985028f9bb48ad779"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "7d2b7ae190991b1287c7f9271b9dbf01"
server_key_iv: "efe120d4f8d9673c3dab66839d216845"
server_key_version: "encryptionkey@0"
I20260806 06:44:03.240684 29894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:03.252626 29894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:03.254698 29894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:03.254853 29894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:03.255066 29894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:03.255535 29946 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:44:03.257347 29894 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:44:03.257418 29894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:44:03.257462 29894 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:44:03.258080 29894 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:44:03.258132 29894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:03.258188 29946 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap starting.
I20260806 06:44:03.265458 29894 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:44741
I20260806 06:44:03.265594 30055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:44741 every 8 connection(s)
I20260806 06:44:03.265875 29894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:44:03.269577 29946 log.cc:824] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:03.272140 29946 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:03.272439 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 29894
I20260806 06:44:03.272452 29946 tablet_bootstrap.cc:492] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Bootstrap complete.
I20260806 06:44:03.272706 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:41359
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=43735
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:03.273615 29946 ts_tablet_manager.cc:1397] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260806 06:44:03.274923 29946 raft_consensus.cc:348] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:03.275264 29946 raft_consensus.cc:729] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96dbc7b852b4d0985028f9bb48ad779, State: Initialized, Role: FOLLOWER
I20260806 06:44:03.275456 29946 consensus_queue.cc:260] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:03.275743 29946 ts_tablet_manager.cc:1428] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260806 06:44:03.332412 30058 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:44:03.266606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:46897)
0806 06:44:03.266794 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.266828 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:03.267439 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:03.332207 (+ 64768us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
W20260806 06:44:03.332666 30056 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:37131 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:37131: Client connection negotiation failed: client connection to 127.14.51.126:37131: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:44:03.333032 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.332905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:34005)
0806 06:44:03.332960 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.333006 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:44:03.333349 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.333258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:53009)
0806 06:44:03.333302 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.333331 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:44:03.333536 30056 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:44:03.358675 30060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:03.358894 30060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:03.358935 30060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:03.360134 30060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:03.360194 30060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:03.360240 30060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:44:03.361703 30060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:41359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=43735
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:03.361961 30060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:03.362175 30060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:03.367166 30069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.367231 30066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.367228 30067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:03.367352 30060 server_base.cc:1034] running on GCE node
I20260806 06:44:03.367580 30060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:03.367877 30060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:03.369065 30060 hybrid_clock.cc:648] HybridClock initialized: now 1785998643369047 us; error 29 us; skew 500 ppm
Aug 06 06:44:03 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:03.371654 30060 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:44:03.372056 30060 webserver.cc:460] Webserver started at http://127.14.51.66:43735/ using document root <none> and password file <none>
I20260806 06:44:03.372277 30060 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:03.372342 30060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:03.373617 30060 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:03.374198 30076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.374365 30060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.374459 30060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "15dcbf00214343dc950b01a4e85d4be9"
format_stamp: "Formatted at 2026-08-06 06:44:01 on dist-test-slave-1xrh"
server_key: "2be880aec1551d5cc78cb518e4982c7b"
server_key_iv: "c3842503fa24913a99ce694349acfc56"
server_key_version: "encryptionkey@0"
I20260806 06:44:03.374758 30060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:03.405989 30060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:03.408322 30060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:03.408483 30060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:03.408712 30060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:03.409005 30060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:03.409062 30060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.409103 30060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:03.409132 30060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.414569 30060 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:41359
I20260806 06:44:03.414633 30189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:41359 every 8 connection(s)
I20260806 06:44:03.414947 30060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:44:03.416028 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.415663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:43405)
0806 06:44:03.415808 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.415889 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
W20260806 06:44:03.416431 30190 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:37131 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:37131: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
I20260806 06:44:03.416690 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.416601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:38801)
0806 06:44:03.416648 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.416674 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:44:03.416987 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.416914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:36117)
0806 06:44:03.416951 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.416973 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:44:03.417112 30190 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:44:03.420256 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30060
I20260806 06:44:03.420517 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:42705
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:37131
--webserver_port=39761
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:03.504815 30193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:03.504999 30193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:03.505033 30193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:03.506138 30193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:03.506177 30193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:03.506225 30193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:44:03.507712 30193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:42705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=39761
--tserver_master_addrs=127.14.51.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:03.507920 30193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:03.508136 30193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:03.512756 30199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.512764 30198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:03.512769 30201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:03.513031 30193 server_base.cc:1034] running on GCE node
I20260806 06:44:03.513234 30193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:03.513464 30193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:03.514602 30193 hybrid_clock.cc:648] HybridClock initialized: now 1785998643514584 us; error 35 us; skew 500 ppm
Aug 06 06:44:03 dist-test-slave-1xrh krb5kdc[28734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:03.517252 30193 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:44:03.517715 30193 webserver.cc:460] Webserver started at http://127.14.51.67:39761/ using document root <none> and password file <none>
I20260806 06:44:03.517896 30193 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:03.517935 30193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:03.519083 30193 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:03.519639 30208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.519800 30193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.519893 30193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "bb1c9b5a160a4116b54d7f9277b21f36"
format_stamp: "Formatted at 2026-08-06 06:44:02 on dist-test-slave-1xrh"
server_key: "fc83682e9390fd6b94e67b78e447a0d5"
server_key_iv: "073032af5736b9669a2ded589b2f71b3"
server_key_version: "encryptionkey@0"
I20260806 06:44:03.520184 30193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:03.533568 30193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:03.535418 30193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:03.535560 30193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:03.535765 30193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:03.536037 30193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:03.536094 30193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.536132 30193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:03.536157 30193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:03.542075 30193 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:42705
I20260806 06:44:03.542114 30322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:42705 every 8 connection(s)
I20260806 06:44:03.542423 30193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:44:03.543394 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.543000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:55507)
0806 06:44:03.543177 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.543261 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
W20260806 06:44:03.543758 30323 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:37131 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:37131: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
I20260806 06:44:03.544071 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.543970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:37583)
0806 06:44:03.544012 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.544054 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:44:03.544412 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:03.544343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:38549)
0806 06:44:03.544376 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:03.544398 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:44:03.544549 30323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:44:03.545398 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30193
I20260806 06:44:04.334209 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.333961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:41233)
0806 06:44:04.334108 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:04.334144 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260806 06:44:04.411473 30328 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:04.411664 30328 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:04.412096 30328 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
I20260806 06:44:04.417685 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.417461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:33849)
0806 06:44:04.417591 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:04.417621 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":40,"threads_started":1}
Aug 06 06:44:04 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.67@KRBTEST.COM
Aug 06 06:44:04 dist-test-slave-1xrh krb5kdc[28734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.66@KRBTEST.COM
I20260806 06:44:04.424204 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.412248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:42705 (local address 127.14.51.65:57239)
0806 06:44:04.412302 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:04.412313 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:04.412368 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:04.413416 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:04.413420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:04.413830 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:04.414028 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:04.414034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.415266 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.415268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:04.416274 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:04.416277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.417191 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.417192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:04.417318 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:04.418105 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:04.418117 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:04.419978 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:04.422492 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:04.422498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:04.422500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:04.422707 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:04.422974 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:04.422975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:04.422975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:04.423104 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:04.423651 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:04.423656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:04.423816 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 06:44:04.423941 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:44:04.424106 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":8}
I20260806 06:44:04.424244 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.412392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:57239 (local address 127.14.51.67:42705)
0806 06:44:04.412564 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:04.412575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:04.412597 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:44:04.413169 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:04.413172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:04.413229 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:04.413342 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:04.414181 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.415165 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.416374 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.417085 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.417297 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:04.420100 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:04.420105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:04.420106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:04.420118 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:04.422407 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:04.422783 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:04.422787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:04.422787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:04.422906 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:04.423198 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:04.423199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:04.423199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:04.423575 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:04.423658 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:04.423999 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 06:44:04.424137 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260806 06:44:04.424355 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.412248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:41359 (local address 127.14.51.65:36493)
0806 06:44:04.412423 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:04.412437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:04.412509 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:04.413423 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:04.413424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:04.413830 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:04.414028 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:04.414034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.415717 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.415718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:04.416422 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:04.416423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.417193 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.417194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:04.417318 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:04.418174 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:04.418180 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:04.420496 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:04.422825 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:04.422826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:04.422827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:04.423024 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:04.423399 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:04.423400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:04.423400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:04.423500 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:04.423955 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:04.423956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:04.424007 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:44:04.424101 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:44:04.424212 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":28,"thread_start_us":80,"threads_started":1}
I20260806 06:44:04.424429 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.412419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:36493 (local address 127.14.51.66:41359)
0806 06:44:04.412600 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:04.412610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:04.412669 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:44:04.413219 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:04.413222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:04.413274 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:04.413442 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:04.414203 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.415617 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.416495 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:04.417100 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:04.417218 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:04.420630 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:04.420635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:04.420636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:04.420649 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:04.422722 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:04.423116 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:04.423119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:04.423138 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:04.423257 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:04.423568 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:04.423569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:04.423569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:04.423875 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:04.423972 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:04.424166 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:44:04.424306 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":58,"threads_started":1}
W20260806 06:44:04.425423 29941 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
W20260806 06:44:04.425511 29942 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:04.425556 29942 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96dbc7b852b4d0985028f9bb48ad779; no voters: 15dcbf00214343dc950b01a4e85d4be9, bb1c9b5a160a4116b54d7f9277b21f36
I20260806 06:44:04.425670 30328 raft_consensus.cc:2726] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:44:04.545071 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:04.544878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:42877)
0806 06:44:04.544992 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:04.545037 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260806 06:44:05.334983 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:05.334769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:53541)
0806 06:44:05.334909 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:05.334939 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260806 06:44:05.418337 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:05.418136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:58579)
0806 06:44:05.418246 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:05.418285 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260806 06:44:05.545676 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:05.545484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:32769)
0806 06:44:05.545592 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:05.545625 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260806 06:44:06.335604 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:06.335417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:44729)
0806 06:44:06.335528 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:06.335560 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260806 06:44:06.418958 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:06.418766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:53411)
0806 06:44:06.418889 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:06.418922 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260806 06:44:06.494349 30339 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:06.494438 30339 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:06.494649 30339 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
W20260806 06:44:06.495011 29941 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
W20260806 06:44:06.495093 29942 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:06.495196 29942 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96dbc7b852b4d0985028f9bb48ad779; no voters: 15dcbf00214343dc950b01a4e85d4be9, bb1c9b5a160a4116b54d7f9277b21f36
I20260806 06:44:06.495286 30339 raft_consensus.cc:2726] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:44:06.546295 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:06.546097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:40091)
0806 06:44:06.546225 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:06.546255 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260806 06:44:07.336236 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:07.335989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:46391)
0806 06:44:07.336136 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:07.336172 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260806 06:44:07.419524 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:07.419325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:42935)
0806 06:44:07.419453 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:07.419488 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260806 06:44:07.546929 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:07.546743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:38019)
0806 06:44:07.546859 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:07.546891 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260806 06:44:08.336946 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:08.336730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:32787)
0806 06:44:08.336863 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:08.336899 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260806 06:44:08.420112 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:08.419937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:38051)
0806 06:44:08.420048 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:08.420076 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260806 06:44:08.547590 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:08.547405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:51557)
0806 06:44:08.547520 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:08.547553 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260806 06:44:09.337592 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:09.337387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:50203)
0806 06:44:09.337506 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:09.337537 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260806 06:44:09.420758 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:09.420512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:60737)
0806 06:44:09.420676 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:09.420711 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260806 06:44:09.463798 30349 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:09.463958 30349 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:09.464116 30349 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
W20260806 06:44:09.464512 29941 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
W20260806 06:44:09.464592 29942 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:09.464690 29942 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96dbc7b852b4d0985028f9bb48ad779; no voters: 15dcbf00214343dc950b01a4e85d4be9, bb1c9b5a160a4116b54d7f9277b21f36
I20260806 06:44:09.464775 30349 raft_consensus.cc:2726] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:44:09.548213 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:09.548009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:51401)
0806 06:44:09.548130 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:09.548175 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260806 06:44:10.338263 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:10.338056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:49897)
0806 06:44:10.338181 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:10.338217 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260806 06:44:10.421458 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:10.421210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:33221)
0806 06:44:10.421361 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:10.421415 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260806 06:44:10.548812 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:10.548631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:52359)
0806 06:44:10.548743 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:10.548776 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260806 06:44:11.339038 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:11.338829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:48913)
0806 06:44:11.338955 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:11.338991 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260806 06:44:11.422076 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:11.421892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:49369)
0806 06:44:11.422004 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:11.422036 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260806 06:44:11.549445 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:11.549239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:46887)
0806 06:44:11.549366 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:11.549400 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260806 06:44:12.339814 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:12.339576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:42763)
0806 06:44:12.339728 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:12.339766 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260806 06:44:12.422770 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:12.422513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:54083)
0806 06:44:12.422696 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:12.422732 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260806 06:44:12.449019 30360 raft_consensus.cc:482] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:12.449108 30360 raft_consensus.cc:504] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15dcbf00214343dc950b01a4e85d4be9" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 41359 } } peers { permanent_uuid: "bb1c9b5a160a4116b54d7f9277b21f36" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 42705 } } peers { permanent_uuid: "b96dbc7b852b4d0985028f9bb48ad779" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 44741 } }
I20260806 06:44:12.449316 30360 leader_election.cc:288] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359), bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705)
W20260806 06:44:12.449674 29942 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb1c9b5a160a4116b54d7f9277b21f36 (127.14.51.67:42705): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
W20260806 06:44:12.449787 29941 leader_election.cc:341] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15dcbf00214343dc950b01a4e85d4be9 (127.14.51.66:41359): Not found: Tablet not found: 4f4261028c3247a69d9460e0b0db5f03
I20260806 06:44:12.449860 29941 leader_election.cc:302] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b96dbc7b852b4d0985028f9bb48ad779; no voters: 15dcbf00214343dc950b01a4e85d4be9, bb1c9b5a160a4116b54d7f9277b21f36
I20260806 06:44:12.449954 30360 raft_consensus.cc:2726] T 4f4261028c3247a69d9460e0b0db5f03 P b96dbc7b852b4d0985028f9bb48ad779 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:44:12.550148 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:12.549960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:45091)
0806 06:44:12.550078 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:12.550111 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260806 06:44:13.340528 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:13.340308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:35561)
0806 06:44:13.340451 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:13.340481 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 06:44:13.423496 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:13.423217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:48319)
0806 06:44:13.423414 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:13.423453 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260806 06:44:13.550917 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:13.550730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:44861)
0806 06:44:13.550849 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:13.550882 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260806 06:44:14.341352 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:14.341000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:56675)
0806 06:44:14.341268 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:14.341305 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":71,"threads_started":1}
I20260806 06:44:14.424243 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:14.424035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:34077)
0806 06:44:14.424163 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:14.424199 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260806 06:44:14.551653 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:14.551429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:51475)
0806 06:44:14.551580 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:14.551613 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260806 06:44:15.342110 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:15.341895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:35525)
0806 06:44:15.342032 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:15.342067 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260806 06:44:15.424934 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:15.424730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:42333)
0806 06:44:15.424855 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:15.424891 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260806 06:44:15.552263 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:15.552060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:53075)
0806 06:44:15.552191 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:15.552225 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 06:44:16.342924 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:16.342671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:42969)
0806 06:44:16.342834 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:16.342870 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":55,"threads_started":1}
I20260806 06:44:16.425655 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:16.425445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:47303)
0806 06:44:16.425572 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:16.425609 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260806 06:44:16.552866 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:16.552683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:39377)
0806 06:44:16.552795 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:16.552826 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260806 06:44:17.343559 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:17.343336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:59757)
0806 06:44:17.343465 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:17.343497 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260806 06:44:17.426303 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:17.426110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:44475)
0806 06:44:17.426228 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:17.426263 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260806 06:44:17.553507 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:17.553309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:55579)
0806 06:44:17.553433 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:17.553468 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260806 06:44:18.344264 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:18.344032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.65:38623)
0806 06:44:18.344185 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:18.344219 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260806 06:44:18.426978 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:18.426781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.66:54963)
0806 06:44:18.426906 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:18.426939 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260806 06:44:18.550078 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 29894
I20260806 06:44:18.554239 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:18.554026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:37131 (local address 127.14.51.67:59385)
0806 06:44:18.554167 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:18.554198 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260806 06:44:18.556347 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30060
I20260806 06:44:18.560956 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30193
2026-08-06T06:44:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18189 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:44:18 dist-test-slave-1xrh krb5kdc[30383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:44:18 dist-test-slave-1xrh krb5kdc[30383](info): set up 2 sockets
Aug 06 06:44:18 dist-test-slave-1xrh krb5kdc[30383](info): commencing operation
krb5kdc: starting...
W20260806 06:44:19.626696 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:44:19 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:44:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:44:19.743777 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:19.842049 30396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:19.842255 30396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:19.842276 30396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:19.843358 30396 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:44:19.843403 30396 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:44:19.843416 30396 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:44:19.843428 30396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:19.843441 30396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:19.843453 30396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:19.843465 30396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:19.843492 30396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:19.844873 30396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:19.845069 30396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:19.845219 30396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:19.849905 30404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:19.849889 30401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:19.850111 30396 server_base.cc:1034] running on GCE node
W20260806 06:44:19.850278 30402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:19.850544 30396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:19.850775 30396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:19.851960 30396 hybrid_clock.cc:648] HybridClock initialized: now 1785998659851944 us; error 41 us; skew 500 ppm
Aug 06 06:44:19 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998659, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:19.854384 30396 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:19.854748 30396 webserver.cc:460] Webserver started at http://127.14.51.126:45901/ using document root <none> and password file <none>
I20260806 06:44:19.854939 30396 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:19.854979 30396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:19.855073 30396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:19.856119 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "83af52de4a7d4ebdbdac49346da8133b"
format_stamp: "Formatted at 2026-08-06 06:44:19 on dist-test-slave-1xrh"
server_key: "86d1b698947bfc2f014ba9aee3972a55"
server_key_iv: "a8cb8220ba0bbc1ffd37359c98402cff"
server_key_version: "encryptionkey@0"
I20260806 06:44:19.856426 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "83af52de4a7d4ebdbdac49346da8133b"
format_stamp: "Formatted at 2026-08-06 06:44:19 on dist-test-slave-1xrh"
server_key: "86d1b698947bfc2f014ba9aee3972a55"
server_key_iv: "a8cb8220ba0bbc1ffd37359c98402cff"
server_key_version: "encryptionkey@0"
I20260806 06:44:19.857676 30396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:44:19.858436 30412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:19.858644 30396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:19.858736 30396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "83af52de4a7d4ebdbdac49346da8133b"
format_stamp: "Formatted at 2026-08-06 06:44:19 on dist-test-slave-1xrh"
server_key: "86d1b698947bfc2f014ba9aee3972a55"
server_key_iv: "a8cb8220ba0bbc1ffd37359c98402cff"
server_key_version: "encryptionkey@0"
I20260806 06:44:19.858811 30396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:19.873690 30396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:19.875411 30396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:19.875546 30396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:19.879525 30396 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:34719
I20260806 06:44:19.879580 30465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:34719 every 8 connection(s)
I20260806 06:44:19.879911 30396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:19.880730 30466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:19.883186 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap starting.
I20260806 06:44:19.883772 30466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:19.884124 30466 log.cc:824] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:19.884773 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: No bootstrap required, opened a new log
I20260806 06:44:19.886317 30466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:19.886467 30466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:19.886523 30466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Initialized, Role: FOLLOWER
I20260806 06:44:19.886686 30466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:19.886766 30466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:19.886814 30466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:19.886862 30466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:19.887383 30466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:19.887514 30466 leader_election.cc:302] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83af52de4a7d4ebdbdac49346da8133b; no voters:
I20260806 06:44:19.887703 30466 leader_election.cc:288] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:44:19.887745 30469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:19.887907 30469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 LEADER]: Becoming Leader. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Running, Role: LEADER
I20260806 06:44:19.887954 30466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:19.888258 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30396
I20260806 06:44:19.888360 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:44:19.888563 14541 external_mini_cluster.cc:1428] Setting key acfb9cb2be51d6052b618384c9bd007f
I20260806 06:44:19.888082 30469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:19.889423 30470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:44:19.889514 30470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:19.890107 30471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83af52de4a7d4ebdbdac49346da8133b. Latest consensus state: current_term: 1 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:44:19.890234 30471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
W20260806 06:44:19.890254 30486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:44:19.890338 30486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:44:19.894196 30489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:19.894390 30489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:44:19 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:19.895843 30489 catalog_manager.cc:1347] Generated new cluster ID: 8c76e8d37fe04fd790f2dbb1f76e657f
I20260806 06:44:19.895916 30489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:44:19.901163 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:19.889257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.14.51.126:34719)
0806 06:44:19.889981 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:19.889990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:19.890073 (+ 83us) server_negotiation.cc:407] Connection header received
0806 06:44:19.890186 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:19.890193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:19.890250 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:19.890388 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:19.891176 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:19.892079 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:19.893158 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:19.893550 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:19.893626 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:19.896478 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:19.896482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:19.896483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:19.896494 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:19.899150 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:19.899559 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:19.899562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:19.899562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:19.899666 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:19.899873 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:19.899873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:19.899874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:19.900163 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:19.900294 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:19.900827 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 06:44:19.900960 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":76,"threads_started":1}
I20260806 06:44:19.905181 30489 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:44:19.905869 30489 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:44:19.913834 30489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Generated new TSK 0
I20260806 06:44:19.914045 30489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:44:19.945349 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:20.025652 30493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:20.025852 30493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:20.025874 30493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:20.027069 30493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:20.027112 30493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:20.027170 30493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:44:20.028644 30493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:20.028880 30493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:20.029126 30493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:20.034309 30498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:20.034404 30493 server_base.cc:1034] running on GCE node
W20260806 06:44:20.034340 30501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:20.034350 30499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:20.034703 30493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:20.034950 30493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:20.036118 30493 hybrid_clock.cc:648] HybridClock initialized: now 1785998660036075 us; error 81 us; skew 500 ppm
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:20.038702 30493 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:44:20.039188 30493 webserver.cc:460] Webserver started at http://127.14.51.65:33211/ using document root <none> and password file <none>
I20260806 06:44:20.039422 30493 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:20.039494 30493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:20.039652 30493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:20.040657 30493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "ae923b4850114eb1a39eb5108fab624b"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "e4c31f8a5c0600cc1bd4064b7b9f09c8"
server_key_iv: "32043d8e9bf35293249bf11dade73aac"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.041018 30493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "ae923b4850114eb1a39eb5108fab624b"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "e4c31f8a5c0600cc1bd4064b7b9f09c8"
server_key_iv: "32043d8e9bf35293249bf11dade73aac"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.042296 30493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:44:20.043004 30508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.043177 30493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:44:20.043273 30493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "ae923b4850114eb1a39eb5108fab624b"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "e4c31f8a5c0600cc1bd4064b7b9f09c8"
server_key_iv: "32043d8e9bf35293249bf11dade73aac"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.043380 30493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:20.062484 30493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:20.064818 30493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:20.064981 30493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:20.065212 30493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:20.065522 30493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:20.065577 30493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.065609 30493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:20.065636 30493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.071324 30493 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:45499
I20260806 06:44:20.071396 30625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:45499 every 8 connection(s)
I20260806 06:44:20.071703 30493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:20.080137 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30493
I20260806 06:44:20.080281 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:44:20.080489 14541 external_mini_cluster.cc:1428] Setting key cee935a0762c2ae631fe2c6151b523e2
I20260806 06:44:20.080574 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.072421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:41487 (local address 127.14.51.126:34719)
0806 06:44:20.072498 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.072500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.072675 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:44:20.073125 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.073126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.073140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.073187 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:20.074045 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.074543 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.075523 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.075712 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.075760 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.077788 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:20.077789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:20.077789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:20.077795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:20.079265 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.079642 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.079643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.079643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.079694 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.079893 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.079894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.079894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.079988 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:20.080067 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.080384 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.080480 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:44:20.080690 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.072384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:41487)
0806 06:44:20.072540 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.072595 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.073045 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.073245 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.073249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.073796 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:20.073973 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.073977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.074618 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.074619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.075447 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.075449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.075778 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.075779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.075890 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.076294 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:20.076301 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:20.077699 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:20.079342 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.079347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.079349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.079542 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.079749 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.079749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.079750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.079840 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.080082 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:20.080089 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:20.080216 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.080327 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.080479 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260806 06:44:20.081169 30626 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:20.081252 30626 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:20.081437 30626 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:20.081986 30429 ts_manager.cc:194] Registered new tserver with Master: ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499)
I20260806 06:44:20.082612 30429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:41487
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:44:20.106272 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:20.185611 30631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:20.185832 30631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:20.185869 30631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:20.186968 30631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:20.187033 30631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:20.187076 30631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:44:20.188648 30631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:20.188894 30631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:20.189190 30631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:20.194159 30636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:20.194144 30639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:20.194227 30631 server_base.cc:1034] running on GCE node
W20260806 06:44:20.194144 30637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:20.194842 30631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:20.195091 30631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:20.196300 30631 hybrid_clock.cc:648] HybridClock initialized: now 1785998660196282 us; error 30 us; skew 500 ppm
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:20.198819 30631 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:44:20.199287 30631 webserver.cc:460] Webserver started at http://127.14.51.66:34563/ using document root <none> and password file <none>
I20260806 06:44:20.199505 30631 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:20.199548 30631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:20.199673 30631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:20.200623 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "c3748c6bf8e64b97b8145df3b961ece6"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "41d22edd33b94b5482e766cf474e688d"
server_key_iv: "a1fb2bf82eeac2434e83459781f527d5"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.200976 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "c3748c6bf8e64b97b8145df3b961ece6"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "41d22edd33b94b5482e766cf474e688d"
server_key_iv: "a1fb2bf82eeac2434e83459781f527d5"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.202196 30631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:44:20.202847 30646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.203163 30631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:20.203261 30631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "c3748c6bf8e64b97b8145df3b961ece6"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "41d22edd33b94b5482e766cf474e688d"
server_key_iv: "a1fb2bf82eeac2434e83459781f527d5"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.203332 30631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:20.213965 30631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:20.216173 30631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:20.216331 30631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:20.216544 30631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:20.216840 30631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:20.216873 30631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.216913 30631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:20.216948 30631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.222494 30631 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:33899
I20260806 06:44:20.222572 30764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:33899 every 8 connection(s)
I20260806 06:44:20.222853 30631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:20.230381 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30631
I20260806 06:44:20.230508 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:44:20.230722 14541 external_mini_cluster.cc:1428] Setting key 6bf804f71993617ea8cd4ce56d6442a7
I20260806 06:44:20.231662 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.223585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44859 (local address 127.14.51.126:34719)
0806 06:44:20.223647 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.223649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.223798 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:44:20.224264 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.224265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.224274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.224330 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:20.225207 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.225699 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.226627 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.226851 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.226910 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.228956 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:20.228958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:20.228958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:20.228964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:20.230124 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.230557 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.230558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.230559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.230632 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.230936 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.230937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.230937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.231105 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:20.231206 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.231490 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.231592 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:44:20.231880 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.223503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:44859)
0806 06:44:20.223681 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.223719 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.224183 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.224381 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.224384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.224910 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:20.225096 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.225100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.225783 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.225784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.226548 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.226550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.226928 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.226929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.227022 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.227507 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:20.227515 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:20.228868 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:20.230199 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.230204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.230206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.230418 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.230710 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.230711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.230711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.230854 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.231199 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:20.231205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:20.231322 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.231451 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.231628 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260806 06:44:20.232337 30765 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:20.232451 30765 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:20.232616 30765 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:20.233011 30429 ts_manager.cc:194] Registered new tserver with Master: c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899)
I20260806 06:44:20.233429 30429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:44859
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:44:20.257393 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:20.339459 30770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:20.339639 30770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:20.339661 30770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:20.340740 30770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:20.340780 30770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:20.340806 30770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:44:20.342227 30770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:20.342393 30770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:20.342553 30770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:20.347404 30778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:20.347383 30776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:20.347384 30775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:20.347688 30770 server_base.cc:1034] running on GCE node
I20260806 06:44:20.347925 30770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:20.348156 30770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:20.349301 30770 hybrid_clock.cc:648] HybridClock initialized: now 1785998660349284 us; error 39 us; skew 500 ppm
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:20.351856 30770 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:44:20.352274 30770 webserver.cc:460] Webserver started at http://127.14.51.67:34639/ using document root <none> and password file <none>
I20260806 06:44:20.352459 30770 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:20.352497 30770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:20.352584 30770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:44:20.353546 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "53a57cfa67e6334af5e69751be2b6ef4"
server_key_iv: "e8364eb4fc596a70571be22a38ab4144"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.353839 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "53a57cfa67e6334af5e69751be2b6ef4"
server_key_iv: "e8364eb4fc596a70571be22a38ab4144"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.355361 30770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:44:20.356056 30785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.356246 30770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:44:20.356331 30770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "53a57cfa67e6334af5e69751be2b6ef4"
server_key_iv: "e8364eb4fc596a70571be22a38ab4144"
server_key_version: "encryptionkey@0"
I20260806 06:44:20.356415 30770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:20.372610 30770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:20.374863 30770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:20.374974 30770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:20.375217 30770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:20.375480 30770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:20.375512 30770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.375533 30770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:20.375546 30770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:20.381209 30770 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37875
I20260806 06:44:20.381294 30901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37875 every 8 connection(s)
I20260806 06:44:20.381598 30770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:44:20.381982 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30770
I20260806 06:44:20.382083 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:44:20.382296 14541 external_mini_cluster.cc:1428] Setting key 798f56d04dcc1960dfccbd7b940144de
Aug 06 06:44:20 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998660, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:20.390980 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.382356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:39913 (local address 127.14.51.126:34719)
0806 06:44:20.382500 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.382503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.382663 (+ 160us) server_negotiation.cc:407] Connection header received
0806 06:44:20.383371 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.383372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.383389 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.383434 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:20.384296 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.384773 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.385722 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.385917 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.385973 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.387995 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:20.387997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:20.387997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:20.388003 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:20.389585 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.389981 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.389982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.389982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.390035 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.390332 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.390333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.390334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.390462 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:20.390535 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.390849 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.390916 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 06:44:20.391148 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.382315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:39913)
0806 06:44:20.382525 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.382572 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.383276 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.383494 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.383497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.384048 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:20.384222 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.384226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.384848 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.384849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.385642 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.385644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.386001 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.386001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.386096 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.386500 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:20.386508 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:20.387902 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:20.389701 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.389706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.389708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.389905 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.390152 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:20.390153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:20.390153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:20.390259 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:20.390555 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:20.390561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:20.390719 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.390819 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.390939 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260806 06:44:20.391607 30902 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:20.391714 30902 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:20.391882 30902 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:20.392237 30429 ts_manager.cc:194] Registered new tserver with Master: 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
I20260806 06:44:20.392570 30429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:39913
I20260806 06:44:20.393947 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:44:20.401551 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.395733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.14.51.126:34719)
0806 06:44:20.395795 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.395797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.395876 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:44:20.395896 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.395897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.395905 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.395952 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:20.396728 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.397170 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.397901 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.398076 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.398126 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.399266 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:20.399267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:20.399267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:20.399272 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:20.400659 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.400976 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.400977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.400977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.401025 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:20.401174 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:20.401174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:20.401174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:20.401291 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:20.401361 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.401438 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.401495 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:44:20.403304 30429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:44:20.404879 30429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:44:20.411486 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.14.51.66:33899)
0806 06:44:20.407679 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.407683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.407692 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:44:20.407768 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.407770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.407787 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.408077 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:20.408160 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409239 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.410547 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411024 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411114 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411194 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.411369 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.411414 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260806 06:44:20.411477 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33899 (local address 127.0.0.1:32952)
0806 06:44:20.407379 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.407389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.407444 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.407910 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.407913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.407916 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:20.407984 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.407986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409348 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.409349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.410463 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.410464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411117 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411141 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.411231 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411245 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.411345 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.411402 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":30,"thread_start_us":51,"threads_started":1}
I20260806 06:44:20.411654 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.14.51.67:37875)
0806 06:44:20.408122 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.408126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.408136 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:44:20.408167 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.408169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.408191 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.408276 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:20.408394 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409039 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.410488 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411176 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411231 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411275 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.411526 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.411579 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":97,"threads_started":1}
I20260806 06:44:20.411654 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.14.51.65:45499)
0806 06:44:20.407824 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.407829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.407842 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:44:20.407886 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.407888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.407903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.407982 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:20.408107 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409258 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.410400 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411107 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411201 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411261 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.411526 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.411572 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":97,"threads_started":1}
I20260806 06:44:20.411738 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:45499 (local address 127.0.0.1:32936)
0806 06:44:20.407755 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.407765 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.407819 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.407987 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.407988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.407990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:20.408037 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.408039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409369 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.409370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.410312 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.410315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411300 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.411385 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411390 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.411440 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.411482 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":24,"thread_start_us":69,"threads_started":1}
I20260806 06:44:20.411934 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.407153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37875 (local address 127.0.0.1:46954)
0806 06:44:20.407255 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.407295 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.407364 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.408276 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.408277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.408279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:20.408324 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.408326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.409140 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.409141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.410413 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.410415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.411304 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.411304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.411385 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.411389 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.411440 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.411481 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260806 06:44:20.412467 30559 tablet_service.cc:1467] Processing CreateTablet for tablet 298c47ab04524076b458b2921985d80e (DEFAULT_TABLE table=test-table [id=59da35198c2d4572b95a566502272179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:20.412577 30697 tablet_service.cc:1467] Processing CreateTablet for tablet 298c47ab04524076b458b2921985d80e (DEFAULT_TABLE table=test-table [id=59da35198c2d4572b95a566502272179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:20.412613 30835 tablet_service.cc:1467] Processing CreateTablet for tablet 298c47ab04524076b458b2921985d80e (DEFAULT_TABLE table=test-table [id=59da35198c2d4572b95a566502272179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:20.412854 30559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298c47ab04524076b458b2921985d80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:20.412922 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298c47ab04524076b458b2921985d80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:20.412922 30835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298c47ab04524076b458b2921985d80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:20.415478 30916 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Bootstrap starting.
I20260806 06:44:20.416184 30917 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Bootstrap starting.
I20260806 06:44:20.416275 30916 tablet_bootstrap.cc:654] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:20.416569 30918 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Bootstrap starting.
I20260806 06:44:20.416934 30917 tablet_bootstrap.cc:654] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:20.416913 30916 log.cc:824] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:20.417214 30918 tablet_bootstrap.cc:654] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:20.417536 30917 log.cc:824] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:20.417740 30918 log.cc:824] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:20.417775 30916 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: No bootstrap required, opened a new log
I20260806 06:44:20.417837 30916 ts_tablet_manager.cc:1397] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:20.418478 30918 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: No bootstrap required, opened a new log
I20260806 06:44:20.418486 30917 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: No bootstrap required, opened a new log
I20260806 06:44:20.418545 30917 ts_tablet_manager.cc:1397] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:20.418545 30918 ts_tablet_manager.cc:1397] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:44:20.419627 30916 raft_consensus.cc:348] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.419797 30916 raft_consensus.cc:374] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:20.419863 30916 raft_consensus.cc:729] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae923b4850114eb1a39eb5108fab624b, State: Initialized, Role: FOLLOWER
I20260806 06:44:20.419988 30916 consensus_queue.cc:260] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.420271 30916 ts_tablet_manager.cc:1428] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:44:20.420401 30626 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
I20260806 06:44:20.420480 30917 raft_consensus.cc:348] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.420476 30918 raft_consensus.cc:348] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.420653 30918 raft_consensus.cc:374] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:20.420653 30917 raft_consensus.cc:374] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:20.420699 30917 raft_consensus.cc:729] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d8a9401f1a2475e879c366eb1e9ed15, State: Initialized, Role: FOLLOWER
I20260806 06:44:20.420699 30918 raft_consensus.cc:729] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3748c6bf8e64b97b8145df3b961ece6, State: Initialized, Role: FOLLOWER
I20260806 06:44:20.420822 30918 consensus_queue.cc:260] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.420851 30917 consensus_queue.cc:260] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.421116 30917 ts_tablet_manager.cc:1428] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:44:20.421124 30918 ts_tablet_manager.cc:1428] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:44:20.421219 30765 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
I20260806 06:44:20.421213 30902 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
W20260806 06:44:20.473508 30766 tablet.cc:2369] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:44:20.476513 30922 raft_consensus.cc:482] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:20.476650 30922 raft_consensus.cc:504] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.477017 30922 leader_election.cc:288] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875), c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899)
I20260806 06:44:20.481025 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.477320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:53095 (local address 127.14.51.66:33899)
0806 06:44:20.477368 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.477370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.477489 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:44:20.477517 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.477518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.477527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.477593 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:20.477724 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.478486 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480073 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.480649 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480727 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.480735 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.480925 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.480963 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:44:20.481025 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.477342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:60489 (local address 127.14.51.67:37875)
0806 06:44:20.477405 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.477407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.477415 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:44:20.477441 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.477441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.477451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.477527 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:20.477726 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.478479 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480073 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.480649 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480727 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.480735 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.480921 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.480961 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:44:20.481040 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.477225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37875 (local address 127.14.51.65:60489)
0806 06:44:20.477284 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.477297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.477340 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.477558 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.477559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.477561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:20.477644 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.477646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.478574 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.478574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.479977 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.479980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.480735 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.480821 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.480825 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.480883 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.480934 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":93}
I20260806 06:44:20.481050 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.477238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33899 (local address 127.14.51.65:53095)
0806 06:44:20.477413 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.477427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:20.477473 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:20.477598 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:20.477599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:20.477600 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:20.477656 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.477657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.478574 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.478574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.479977 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:20.479980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.480736 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.480736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:20.480821 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.480826 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:20.480882 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:44:20.480933 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":75,"thread_start_us":117,"threads_started":1}
I20260806 06:44:20.481386 30856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298c47ab04524076b458b2921985d80e" candidate_uuid: "ae923b4850114eb1a39eb5108fab624b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" is_pre_election: true
I20260806 06:44:20.481513 30718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298c47ab04524076b458b2921985d80e" candidate_uuid: "ae923b4850114eb1a39eb5108fab624b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3748c6bf8e64b97b8145df3b961ece6" is_pre_election: true
I20260806 06:44:20.481559 30856 raft_consensus.cc:2454] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae923b4850114eb1a39eb5108fab624b in term 0.
I20260806 06:44:20.481642 30718 raft_consensus.cc:2454] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae923b4850114eb1a39eb5108fab624b in term 0.
I20260806 06:44:20.481827 30511 leader_election.cc:302] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d8a9401f1a2475e879c366eb1e9ed15, ae923b4850114eb1a39eb5108fab624b; no voters:
I20260806 06:44:20.481958 30922 raft_consensus.cc:2781] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:44:20.482054 30922 raft_consensus.cc:482] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:44:20.482091 30922 raft_consensus.cc:3037] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:20.482924 30922 raft_consensus.cc:504] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.483062 30922 leader_election.cc:288] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 1 election: Requested vote from peers 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875), c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899)
I20260806 06:44:20.483215 30856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298c47ab04524076b458b2921985d80e" candidate_uuid: "ae923b4850114eb1a39eb5108fab624b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
I20260806 06:44:20.483306 30856 raft_consensus.cc:3037] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:20.483316 30718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298c47ab04524076b458b2921985d80e" candidate_uuid: "ae923b4850114eb1a39eb5108fab624b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3748c6bf8e64b97b8145df3b961ece6"
I20260806 06:44:20.483408 30718 raft_consensus.cc:3037] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:20.484057 30856 raft_consensus.cc:2454] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae923b4850114eb1a39eb5108fab624b in term 1.
I20260806 06:44:20.484134 30718 raft_consensus.cc:2454] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae923b4850114eb1a39eb5108fab624b in term 1.
I20260806 06:44:20.484256 30511 leader_election.cc:302] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d8a9401f1a2475e879c366eb1e9ed15, ae923b4850114eb1a39eb5108fab624b; no voters:
I20260806 06:44:20.484383 30922 raft_consensus.cc:2781] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:20.484562 30922 raft_consensus.cc:686] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 LEADER]: Becoming Leader. State: Replica: ae923b4850114eb1a39eb5108fab624b, State: Running, Role: LEADER
I20260806 06:44:20.484651 30922 consensus_queue.cc:237] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:20.485461 30427 catalog_manager.cc:5495] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b reported cstate change: term changed from 0 to 1, leader changed from <none> to ae923b4850114eb1a39eb5108fab624b (127.14.51.65). New cstate: current_term: 1 leader_uuid: "ae923b4850114eb1a39eb5108fab624b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } health_report { overall_health: HEALTHY } } }
I20260806 06:44:20.505425 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.503203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.14.51.65:45499)
0806 06:44:20.503264 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:20.503265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:20.503273 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:44:20.503300 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:20.503300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:20.503316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:20.503399 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:20.503511 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.503987 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.504683 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:20.504876 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:20.504925 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:20.505100 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:20.505199 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:44:20.505260 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:20.505290 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:44:20.505307 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:44:20.507156 30718 raft_consensus.cc:1261] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Refusing update from remote peer ae923b4850114eb1a39eb5108fab624b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:20.507154 30856 raft_consensus.cc:1261] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Refusing update from remote peer ae923b4850114eb1a39eb5108fab624b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:20.507500 30922 consensus_queue.cc:1035] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:20.507682 30926 consensus_queue.cc:1035] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:20.509886 30929 mvcc.cc:205] Tried to move back new op lower bound from 7315450513434382336 to 7315450513345630208. Current Snapshot: MvccSnapshot[applied={T|T < 7315450513434382336}]
I20260806 06:44:20.510172 30930 mvcc.cc:205] Tried to move back new op lower bound from 7315450513434382336 to 7315450513345630208. Current Snapshot: MvccSnapshot[applied={T|T < 7315450513434382336}]
I20260806 06:44:20.510406 30931 mvcc.cc:205] Tried to move back new op lower bound from 7315450513434382336 to 7315450513345630208. Current Snapshot: MvccSnapshot[applied={T|T < 7315450513434382336}]
I20260806 06:44:20.512356 30427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:20.512436 30427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:20.513171 30427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Sending DeleteTablet for 3 replicas of tablet 298c47ab04524076b458b2921985d80e
I20260806 06:44:20.513505 30559 tablet_service.cc:1514] Processing DeleteTablet for tablet 298c47ab04524076b458b2921985d80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:20 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:32936
I20260806 06:44:20.513505 30835 tablet_service.cc:1514] Processing DeleteTablet for tablet 298c47ab04524076b458b2921985d80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:20 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46954
I20260806 06:44:20.513543 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet 298c47ab04524076b458b2921985d80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:20 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:32952
I20260806 06:44:20.513712 30940 tablet_replica.cc:330] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: stopping tablet replica
I20260806 06:44:20.513782 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30493
I20260806 06:44:20.513803 30940 raft_consensus.cc:2229] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:20.516269 30940 raft_consensus.cc:2258] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:20.516484 30941 tablet_replica.cc:330] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: stopping tablet replica
I20260806 06:44:20.516774 30941 raft_consensus.cc:2229] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:20.516809 30940 ts_tablet_manager.cc:1905] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:20.516891 30941 raft_consensus.cc:2258] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:20.518584 30940 ts_tablet_manager.cc:1918] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:20.518646 30940 log.cc:1196] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/298c47ab04524076b458b2921985d80e
I20260806 06:44:20.518894 30940 ts_tablet_manager.cc:1939] T 298c47ab04524076b458b2921985d80e P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting consensus metadata
I20260806 06:44:20.519326 30415 catalog_manager.cc:4841] TS 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): tablet 298c47ab04524076b458b2921985d80e (table test-table [id=59da35198c2d4572b95a566502272179]) successfully deleted
I20260806 06:44:20.520116 30941 ts_tablet_manager.cc:1905] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:20.521878 30941 ts_tablet_manager.cc:1918] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:20.521939 30941 log.cc:1196] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/298c47ab04524076b458b2921985d80e
I20260806 06:44:20.522204 30941 ts_tablet_manager.cc:1939] T 298c47ab04524076b458b2921985d80e P c3748c6bf8e64b97b8145df3b961ece6: Deleting consensus metadata
I20260806 06:44:20.522604 30416 catalog_manager.cc:4841] TS c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): tablet 298c47ab04524076b458b2921985d80e (table test-table [id=59da35198c2d4572b95a566502272179]) successfully deleted
W20260806 06:44:20.524207 30413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:45499 (error 108)
I20260806 06:44:20.524549 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30631
I20260806 06:44:20.524734 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:20.524588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:45499 (local address 127.0.0.1:32958)
0806 06:44:20.524653 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:20.524696 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 06:44:20.525208 30413 catalog_manager.cc:4584] TS ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 298c47ab04524076b458b2921985d80e: Network error: Client connection negotiation failed: client connection to 127.14.51.65:45499: connect: Connection refused (error 111)
I20260806 06:44:20.534765 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30770
I20260806 06:44:20.539767 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30396
I20260806 06:44:40.546195 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--webserver_interface=127.14.51.126
--webserver_port=45901
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:44:40.627262 30943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:40.627497 30943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:40.627533 30943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:40.628646 30943 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:44:40.628708 30943 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:44:40.628731 30943 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:44:40.628750 30943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:44:40.628782 30943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:40.628806 30943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:40.628837 30943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:44:40.628868 30943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:44:40.630287 30943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:40.630553 30943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:40.630769 30943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:40.639671 30950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:40.639678 30943 server_base.cc:1034] running on GCE node
W20260806 06:44:40.639684 30949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:40.639683 30952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:40.640509 30943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:40.640779 30943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:40.641925 30943 hybrid_clock.cc:648] HybridClock initialized: now 1785998680641916 us; error 29 us; skew 500 ppm
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:40.649245 30943 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:44:40.649730 30943 webserver.cc:460] Webserver started at http://127.14.51.126:45901/ using document root <none> and password file <none>
I20260806 06:44:40.649968 30943 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:40.650053 30943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:40.651510 30943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:44:40.652189 30959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:40.652355 30943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:40.652482 30943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "83af52de4a7d4ebdbdac49346da8133b"
format_stamp: "Formatted at 2026-08-06 06:44:19 on dist-test-slave-1xrh"
server_key: "86d1b698947bfc2f014ba9aee3972a55"
server_key_iv: "a8cb8220ba0bbc1ffd37359c98402cff"
server_key_version: "encryptionkey@0"
I20260806 06:44:40.652843 30943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:40.674127 30943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:40.675895 30943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:40.676060 30943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:40.680107 30943 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:34719
I20260806 06:44:40.680164 31014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:34719 every 8 connection(s)
I20260806 06:44:40.680574 30943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:44:40.681226 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 30943
I20260806 06:44:40.681540 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:45499
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=33211
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:40.682663 31015 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:44:40.683522 31015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap starting.
I20260806 06:44:40.691817 31015 log.cc:824] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:40.693658 31015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:40.693948 31015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap complete.
I20260806 06:44:40.696374 31015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:40.696599 31015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Initialized, Role: FOLLOWER
I20260806 06:44:40.696764 31015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:40.696825 31015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:44:40.696877 31015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:44:40.696925 31015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:44:40.697922 31015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:40.698006 31015 leader_election.cc:302] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83af52de4a7d4ebdbdac49346da8133b; no voters:
I20260806 06:44:40.698233 31015 leader_election.cc:288] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:44:40.698268 31019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:44:40.698386 31019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 LEADER]: Becoming Leader. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Running, Role: LEADER
I20260806 06:44:40.698499 31015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:44:40.698540 31019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:44:40.698875 31019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83af52de4a7d4ebdbdac49346da8133b. Latest consensus state: current_term: 2 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:44:40.698876 31020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:44:40.699141 31020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:40.699115 31019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
I20260806 06:44:40.699522 31029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:44:40.700343 31029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:44:40.700484 31029 catalog_manager.cc:1259] Loaded cluster ID: 8c76e8d37fe04fd790f2dbb1f76e657f
I20260806 06:44:40.700565 31029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:44:40.702019 31029 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:44:40.702243 31029 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Loaded TSK: 0
I20260806 06:44:40.716655 31029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Generated new TSK 1
I20260806 06:44:40.716727 31029 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:44:40.766796 31017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:40.767014 31017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:40.767050 31017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:40.768163 31017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:40.768226 31017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:40.768273 31017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:44:40.769728 31017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:45499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=33211
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:40.769986 31017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:40.770205 31017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:40.776270 31044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:40.776400 31043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:40.776274 31046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:40.776544 31017 server_base.cc:1034] running on GCE node
I20260806 06:44:40.776731 31017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:40.777036 31017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:40.778220 31017 hybrid_clock.cc:648] HybridClock initialized: now 1785998680778210 us; error 42 us; skew 500 ppm
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:40.781260 31017 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:44:40.781733 31017 webserver.cc:460] Webserver started at http://127.14.51.65:33211/ using document root <none> and password file <none>
I20260806 06:44:40.781975 31017 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:40.782095 31017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:40.783556 31017 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:44:40.784272 31053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:40.784461 31017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:44:40.784559 31017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "ae923b4850114eb1a39eb5108fab624b"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "e4c31f8a5c0600cc1bd4064b7b9f09c8"
server_key_iv: "32043d8e9bf35293249bf11dade73aac"
server_key_version: "encryptionkey@0"
I20260806 06:44:40.785003 31017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:40.813627 31017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:40.816066 31017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:40.816205 31017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:40.816429 31017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:40.816860 31061 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:44:40.818286 31017 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:44:40.818344 31017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:44:40.818377 31017 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:44:40.819206 31017 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:44:40.819250 31017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:40.819309 31061 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Bootstrap starting.
I20260806 06:44:40.825888 31017 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:45499
I20260806 06:44:40.826049 31172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:45499 every 8 connection(s)
I20260806 06:44:40.826336 31017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:44:40.826987 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31017
I20260806 06:44:40.827327 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:33899
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=34563
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:40.830099 31061 log.cc:824] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:40.833627 31061 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:44:40.833973 31061 tablet_bootstrap.cc:492] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Bootstrap complete.
I20260806 06:44:40.835248 31061 ts_tablet_manager.cc:1397] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:40.837278 31061 raft_consensus.cc:348] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:40.837733 31061 raft_consensus.cc:729] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae923b4850114eb1a39eb5108fab624b, State: Initialized, Role: FOLLOWER
I20260806 06:44:40.837914 31061 consensus_queue.cc:260] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:40.838541 31061 ts_tablet_manager.cc:1428] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:44:40.841398 31175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.827005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:34285)
0806 06:44:40.827324 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:40.827376 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:40.828055 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:40.830725 (+ 2670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:40.830729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:40.831047 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:40.831315 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.831321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.832961 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.832962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.834209 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.834211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.834997 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.834998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.835186 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.835850 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:40.835859 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:40.837617 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:40.839804 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.839810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.839812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.840030 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.840313 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.840314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.840314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.840426 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.840787 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:40.840793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:40.840928 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 06:44:40.841042 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:44:40.841211 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":91,"threads_started":1}
I20260806 06:44:40.841398 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.828611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:34285 (local address 127.14.51.126:34719)
0806 06:44:40.830317 (+ 1706us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:40.830326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:40.830365 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:44:40.830501 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:40.830508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:40.830560 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:40.830693 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:40.831449 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.832823 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.834355 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.834904 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.835509 (+ 605us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.837776 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:40.837780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:40.837780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:40.837791 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:40.839697 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.840122 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.840125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.840125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.840234 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.840498 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.840499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.840499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.840706 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:40.840794 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:40.841113 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 06:44:40.841229 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1654,"thread_start_us":94,"threads_started":1}
I20260806 06:44:40.842036 31173 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:40.842131 31173 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:40.842296 31173 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:40.842969 30977 ts_manager.cc:194] Registered new tserver with Master: ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499)
I20260806 06:44:40.844182 30977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:34285
I20260806 06:44:40.845213 31173 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.65@KRBTEST.COM
I20260806 06:44:40.851049 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.843622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.14.51.65:45499)
0806 06:44:40.843771 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:40.843775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:40.843851 (+ 76us) server_negotiation.cc:407] Connection header received
0806 06:44:40.843926 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:40.843928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:40.843942 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:40.844017 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:40.844477 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.845151 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.846115 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.846397 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.846476 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.848356 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:40.848363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:40.848364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:40.848370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:40.849755 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.850182 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.850185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.850185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.850242 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.850455 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.850455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.850456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.850597 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:40.850685 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:40.850875 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:44:40.850962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260806 06:44:40.851050 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.843562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:45499 (local address 127.0.0.1:48934)
0806 06:44:40.843743 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:40.843771 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:40.843851 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:40.844034 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:40.844039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:40.844338 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:40.844385 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.844387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.845246 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.845247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.846016 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.846019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.846477 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.846478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.846564 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.846991 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:40.846995 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:40.848264 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:40.849858 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.849865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.849867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.850089 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.850317 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.850318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.850318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.850384 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.850678 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:40.850684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:40.850733 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:44:40.850825 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:44:40.850945 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260806 06:44:40.851425 31104 tablet_service.cc:1514] Processing DeleteTablet for tablet 298c47ab04524076b458b2921985d80e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:20 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48934
I20260806 06:44:40.851884 31182 tablet_replica.cc:330] stopping tablet replica
I20260806 06:44:40.851989 31182 raft_consensus.cc:2229] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:40.852052 31182 raft_consensus.cc:2258] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:40.852329 31182 ts_tablet_manager.cc:1905] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:40.854720 31182 ts_tablet_manager.cc:1918] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:40.854795 31182 log.cc:1196] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal/wals/298c47ab04524076b458b2921985d80e
I20260806 06:44:40.855018 31182 ts_tablet_manager.cc:1939] T 298c47ab04524076b458b2921985d80e P ae923b4850114eb1a39eb5108fab624b: Deleting consensus metadata
I20260806 06:44:40.855556 30960 catalog_manager.cc:4841] TS ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499): tablet 298c47ab04524076b458b2921985d80e (table test-table [id=59da35198c2d4572b95a566502272179]) successfully deleted
W20260806 06:44:40.913616 31176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:40.913801 31176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:40.913825 31176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:40.914981 31176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:40.915022 31176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:40.915071 31176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:44:40.916507 31176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:33899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=34563
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:40.916738 31176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:40.916950 31176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:40.921686 31187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:40.921730 31191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:40.921697 31189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:40.921967 31176 server_base.cc:1034] running on GCE node
I20260806 06:44:40.922142 31176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:40.922312 31176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:40.923435 31176 hybrid_clock.cc:648] HybridClock initialized: now 1785998680923423 us; error 29 us; skew 500 ppm
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:40.925736 31176 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:44:40.926137 31176 webserver.cc:460] Webserver started at http://127.14.51.66:34563/ using document root <none> and password file <none>
I20260806 06:44:40.926363 31176 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:40.926455 31176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:40.927870 31176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:40.928426 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:40.928597 31176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:44:40.928692 31176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "c3748c6bf8e64b97b8145df3b961ece6"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "41d22edd33b94b5482e766cf474e688d"
server_key_iv: "a1fb2bf82eeac2434e83459781f527d5"
server_key_version: "encryptionkey@0"
I20260806 06:44:40.929008 31176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:40.935021 31176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:40.936695 31176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:40.936873 31176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:40.937103 31176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:40.937366 31176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:40.937424 31176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:40.937461 31176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:40.937489 31176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:40.943610 31176 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:33899
I20260806 06:44:40.943647 31316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:33899 every 8 connection(s)
I20260806 06:44:40.943994 31176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:44:40 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998680, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:40.952864 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.944666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:50783 (local address 127.14.51.126:34719)
0806 06:44:40.944712 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:40.944714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:40.944903 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:44:40.945406 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:40.945408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:40.945423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:40.945483 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:40.946388 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.946882 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.947868 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.948106 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.948194 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.950272 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:40.950274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:40.950274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:40.950282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:40.951500 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.951880 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.951882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.951882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.951943 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:40.952196 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:40.952197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:40.952198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:40.952320 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:40.952389 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:40.952681 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 06:44:40.952775 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:44:40.953006 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:40.944653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:50783)
0806 06:44:40.944791 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:40.944822 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:40.945306 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:40.945551 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:40.945554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:40.946121 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:40.946291 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.946295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.946976 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.946977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.947780 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:40.947782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:40.948203 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:40.948205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:40.948326 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:40.948888 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:40.948895 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:40.950173 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:40.951596 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.951603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.951605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.951807 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.952009 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:40.952010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:40.952011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:40.952123 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:40.952392 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:40.952398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:40.952544 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 06:44:40.952648 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:44:40.952793 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260806 06:44:40.953212 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31176
I20260806 06:44:40.953459 31317 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:40.953508 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:37875
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=34639
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:44:40.953577 31317 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:40.953753 31317 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:40.954164 30977 ts_manager.cc:194] Registered new tserver with Master: c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899)
I20260806 06:44:40.954661 30977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:50783
W20260806 06:44:41.034009 31321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:44:41.034211 31321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:44:41.034233 31321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:44:41.035338 31321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:44:41.035380 31321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:44:41.035405 31321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:44:41.036788 31321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:37875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=34639
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:44:41.037005 31321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:44:41.037240 31321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:44:41.042075 31327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:41.042129 31326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:44:41.042272 31329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:44:41.042291 31321 server_base.cc:1034] running on GCE node
I20260806 06:44:41.042502 31321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:44:41.042683 31321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:44:41.043825 31321 hybrid_clock.cc:648] HybridClock initialized: now 1785998681043814 us; error 28 us; skew 500 ppm
Aug 06 06:44:41 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:44:41.046178 31321 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:44:41.046633 31321 webserver.cc:460] Webserver started at http://127.14.51.67:34639/ using document root <none> and password file <none>
I20260806 06:44:41.046841 31321 fs_manager.cc:359] Metadata directory not provided
I20260806 06:44:41.046919 31321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:44:41.048202 31321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:44:41.048758 31336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:41.048923 31321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:44:41.049050 31321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "53a57cfa67e6334af5e69751be2b6ef4"
server_key_iv: "e8364eb4fc596a70571be22a38ab4144"
server_key_version: "encryptionkey@0"
I20260806 06:44:41.049357 31321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:44:41.061618 31321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:44:41.063443 31321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:44:41.063588 31321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:44:41.063796 31321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:44:41.064124 31321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:44:41.064177 31321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:41.064210 31321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:44:41.064237 31321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:44:41.069837 31321 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37875
I20260806 06:44:41.069871 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37875 every 8 connection(s)
I20260806 06:44:41.070230 31321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:44:41 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:44:41.078472 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.070902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:49497 (local address 127.14.51.126:34719)
0806 06:44:41.070972 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.070974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.071284 (+ 310us) server_negotiation.cc:407] Connection header received
0806 06:44:41.071715 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.071716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.071726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.071775 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:41.072633 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.073097 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.074022 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.074205 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.074254 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.076181 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:41.076183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:41.076183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:41.076190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:41.077413 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.077749 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.077750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.077750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.077796 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.077987 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.077987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.077988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.078086 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:41.078133 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.078348 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.078412 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:44:41.078609 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.070887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:49497)
0806 06:44:41.071064 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.071102 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:41.071634 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:41.071830 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:41.071833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:41.072365 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:44:41.072560 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.072564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.073172 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.073173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.073942 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.073944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.074273 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.074273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.074364 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.074769 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:44:41.074776 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:44:41.076097 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:44:41.077481 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:41.077486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:41.077488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:41.077687 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:41.077851 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:44:41.077852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:44:41.077852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:44:41.077937 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:44:41.078139 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:44:41.078142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:44:41.078240 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 06:44:41.078321 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:44:41.078434 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260806 06:44:41.078984 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31321
I20260806 06:44:41.079172 31456 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34719
I20260806 06:44:41.079262 31456 heartbeater.cc:464] Registering TS with master...
I20260806 06:44:41.079456 31456 heartbeater.cc:511] Master 127.14.51.126:34719 requested a full tablet report, sending...
I20260806 06:44:41.079835 30977 ts_manager.cc:194] Registered new tserver with Master: 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
I20260806 06:44:41.080207 30977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:49497
I20260806 06:44:41.085616 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.079450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.14.51.126:34719)
0806 06:44:41.079489 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.079491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.079709 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:44:41.079771 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.079772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.079782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.079825 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:41.080467 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.080976 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.081695 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.081929 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.082004 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.083218 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:41.083219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:41.083220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:41.083226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:41.084380 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.084745 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.084746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.084746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.084797 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.085012 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.085012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.085013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.085144 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:41.085340 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.085375 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.085494 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:44:41.086333 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:44:41.088686 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.086664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.14.51.126:34719)
0806 06:44:41.086701 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.086702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.086799 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:44:41.086853 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.086854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.086879 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.086950 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:41.087019 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.087474 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.088131 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.088292 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.088341 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.088507 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:41.088550 (+ 43us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 06:44:41.088652 (+ 102us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43376: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:44:41.088754 31177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43376: token expired
W20260806 06:44:41.088969 30907 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.51.126:34719)
I20260806 06:44:41.094209 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.089190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.14.51.126:34719)
0806 06:44:41.089219 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.089221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.089279 (+ 58us) server_negotiation.cc:407] Connection header received
0806 06:44:41.089313 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.089313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.089320 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.089360 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:44:41.089823 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.090242 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.090820 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.090988 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.091035 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.092044 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:44:41.092045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:44:41.092045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:44:41.092051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:44:41.093341 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.093601 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.093601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.093601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.093642 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:44:41.093774 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:44:41.093775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:44:41.093775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:44:41.093847 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:44:41.093905 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.094013 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.094079 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:44:41.095291 30977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:44:41.096879 30977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:44:41.099361 31104 tablet_service.cc:1467] Processing CreateTablet for tablet b57bc425e11c48a5a7352518b523c5ef (DEFAULT_TABLE table=test-table [id=188e325d8a2a4481b14f460f7d45a881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:41.099608 31104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57bc425e11c48a5a7352518b523c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:41.101187 31061 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Bootstrap starting.
I20260806 06:44:41.101532 31061 tablet_bootstrap.cc:654] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:41.102044 31061 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: No bootstrap required, opened a new log
I20260806 06:44:41.102087 31061 ts_tablet_manager.cc:1397] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:44:41.102409 31061 raft_consensus.cc:348] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.102473 31061 raft_consensus.cc:374] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:41.102530 31061 raft_consensus.cc:729] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae923b4850114eb1a39eb5108fab624b, State: Initialized, Role: FOLLOWER
I20260806 06:44:41.102573 31061 consensus_queue.cc:260] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.102667 31061 ts_tablet_manager.cc:1428] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:44:41.102911 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.098838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37875 (local address 127.0.0.1:51034)
0806 06:44:41.098951 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.098960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:41.099000 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:41.099778 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:41.099779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:41.099781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:41.099841 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.099842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.100642 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.100643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.101623 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.101627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.102625 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.102627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.102693 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.102718 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 06:44:41.102791 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:44:41.102828 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260806 06:44:41.102933 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.098929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.14.51.67:37875)
0806 06:44:41.099515 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.099519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.099529 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:44:41.099561 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.099563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.099577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.099668 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:41.099904 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.100555 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.101700 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.102506 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.102608 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.102695 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.102807 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.102857 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":75,"threads_started":1}
I20260806 06:44:41.103144 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.098804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33899 (local address 127.0.0.1:56956)
0806 06:44:41.098932 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.098943 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:41.098993 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:41.099399 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:41.099401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:41.099402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:41.099444 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.099445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.100883 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.100884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.101971 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.101973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.102872 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.102873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.102947 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.102952 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:41.102986 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 06:44:41.103049 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 06:44:41.103511 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.098898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.14.51.66:33899)
0806 06:44:41.099246 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.099250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.099258 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:44:41.099289 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.099291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.099303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.099366 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:41.099592 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.100779 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.102080 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.102779 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.103297 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.103369 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.103397 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.103438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":109,"threads_started":1}
I20260806 06:44:41.103952 31387 tablet_service.cc:1467] Processing CreateTablet for tablet b57bc425e11c48a5a7352518b523c5ef (DEFAULT_TABLE table=test-table [id=188e325d8a2a4481b14f460f7d45a881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:41.104046 31248 tablet_service.cc:1467] Processing CreateTablet for tablet b57bc425e11c48a5a7352518b523c5ef (DEFAULT_TABLE table=test-table [id=188e325d8a2a4481b14f460f7d45a881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:44:41.104274 31387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57bc425e11c48a5a7352518b523c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:41.104374 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57bc425e11c48a5a7352518b523c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:44:41.106989 31465 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Bootstrap starting.
I20260806 06:44:41.106983 31466 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Bootstrap starting.
I20260806 06:44:41.107560 31466 tablet_bootstrap.cc:654] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:41.107584 31465 tablet_bootstrap.cc:654] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Neither blocks nor log segments found. Creating new log.
I20260806 06:44:41.108081 31466 log.cc:824] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:41.108088 31465 log.cc:824] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Log is configured to *not* fsync() on all Append() calls
I20260806 06:44:41.108804 31465 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: No bootstrap required, opened a new log
I20260806 06:44:41.108804 31466 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: No bootstrap required, opened a new log
I20260806 06:44:41.108875 31465 ts_tablet_manager.cc:1397] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:44:41.108875 31466 ts_tablet_manager.cc:1397] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:44:41.110766 31465 raft_consensus.cc:348] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.110766 31466 raft_consensus.cc:348] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.110937 31466 raft_consensus.cc:374] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:41.110937 31465 raft_consensus.cc:374] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:44:41.110975 31465 raft_consensus.cc:729] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d8a9401f1a2475e879c366eb1e9ed15, State: Initialized, Role: FOLLOWER
I20260806 06:44:41.110975 31466 raft_consensus.cc:729] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3748c6bf8e64b97b8145df3b961ece6, State: Initialized, Role: FOLLOWER
I20260806 06:44:41.111091 31466 consensus_queue.cc:260] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.111091 31465 consensus_queue.cc:260] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.111392 31466 ts_tablet_manager.cc:1428] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:44:41.111392 31465 ts_tablet_manager.cc:1428] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:44:41.111414 31317 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
I20260806 06:44:41.111514 31456 heartbeater.cc:503] Master 127.14.51.126:34719 was elected leader, sending a full tablet report...
W20260806 06:44:41.194574 31319 tablet.cc:2369] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:44:41.253976 31470 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:44:41.254098 31470 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.254462 31470 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875), ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499)
I20260806 06:44:41.257675 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.254654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:59271 (local address 127.14.51.67:37875)
0806 06:44:41.254706 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.254708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.254739 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:44:41.254843 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.254845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.254857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.254912 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:41.255084 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.255760 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.256925 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.257336 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.257391 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.257398 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.257562 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.257600 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:44:41.257694 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.254623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37875 (local address 127.14.51.66:59271)
0806 06:44:41.254691 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.254702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:41.254751 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:41.254927 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:41.254929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:41.254931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:41.255010 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.255012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.255928 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.255929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.256813 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.256815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.257413 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.257415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.257491 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.257496 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:44:41.257546 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:44:41.257582 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":4}
I20260806 06:44:41.257826 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.254633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:45499 (local address 127.14.51.66:36259)
0806 06:44:41.254826 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.254837 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:44:41.254900 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:44:41.255086 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:44:41.255087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:44:41.255089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:44:41.255182 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.255183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.255929 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.255930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.256810 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:44:41.256813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.257567 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.257568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:44:41.257633 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.257637 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:44:41.257693 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:44:41.257731 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":42,"thread_start_us":112,"threads_started":1}
I20260806 06:44:41.257850 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.254747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:36259 (local address 127.14.51.65:45499)
0806 06:44:41.254823 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.254825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.254938 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:44:41.254965 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.254966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.254994 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.255060 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:44:41.255241 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.255819 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.256900 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.257458 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.257533 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.257611 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.257720 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.257767 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:44:41.258046 31407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57bc425e11c48a5a7352518b523c5ef" candidate_uuid: "c3748c6bf8e64b97b8145df3b961ece6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" is_pre_election: true
I20260806 06:44:41.258194 31407 raft_consensus.cc:2454] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3748c6bf8e64b97b8145df3b961ece6 in term 0.
I20260806 06:44:41.258420 31201 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6; no voters:
I20260806 06:44:41.258438 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57bc425e11c48a5a7352518b523c5ef" candidate_uuid: "c3748c6bf8e64b97b8145df3b961ece6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae923b4850114eb1a39eb5108fab624b" is_pre_election: true
I20260806 06:44:41.258580 31125 raft_consensus.cc:2454] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3748c6bf8e64b97b8145df3b961ece6 in term 0.
I20260806 06:44:41.258577 31470 raft_consensus.cc:2781] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:44:41.258634 31470 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:44:41.258667 31470 raft_consensus.cc:3037] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:41.259512 31470 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.259670 31470 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [CANDIDATE]: Term 1 election: Requested vote from peers 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875), ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499)
I20260806 06:44:41.259821 31407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57bc425e11c48a5a7352518b523c5ef" candidate_uuid: "c3748c6bf8e64b97b8145df3b961ece6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
I20260806 06:44:41.259903 31407 raft_consensus.cc:3037] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:41.259895 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57bc425e11c48a5a7352518b523c5ef" candidate_uuid: "c3748c6bf8e64b97b8145df3b961ece6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae923b4850114eb1a39eb5108fab624b"
I20260806 06:44:41.259990 31125 raft_consensus.cc:3037] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:44:41.260614 31407 raft_consensus.cc:2454] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3748c6bf8e64b97b8145df3b961ece6 in term 1.
I20260806 06:44:41.260732 31125 raft_consensus.cc:2454] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3748c6bf8e64b97b8145df3b961ece6 in term 1.
I20260806 06:44:41.260823 31201 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6; no voters:
I20260806 06:44:41.260913 31470 raft_consensus.cc:2781] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:44:41.261042 31470 raft_consensus.cc:686] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 1 LEADER]: Becoming Leader. State: Replica: c3748c6bf8e64b97b8145df3b961ece6, State: Running, Role: LEADER
I20260806 06:44:41.261145 31470 consensus_queue.cc:237] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:44:41.261997 30976 catalog_manager.cc:5495] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66). New cstate: current_term: 1 leader_uuid: "c3748c6bf8e64b97b8145df3b961ece6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } health_report { overall_health: UNKNOWN } } }
I20260806 06:44:41.283691 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.281366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.14.51.66:33899)
0806 06:44:41.281423 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:44:41.281425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:44:41.281451 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:44:41.281489 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:44:41.281490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:44:41.281505 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:44:41.281586 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:44:41.281709 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.282221 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.282937 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:44:41.283170 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:44:41.283239 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:44:41.283433 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:44:41.283524 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:44:41.283587 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:44:41.283622 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:44:41.283639 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:44:41.285521 31407 raft_consensus.cc:1261] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Refusing update from remote peer c3748c6bf8e64b97b8145df3b961ece6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:41.285521 31125 raft_consensus.cc:1261] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Refusing update from remote peer c3748c6bf8e64b97b8145df3b961ece6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:44:41.285899 31472 consensus_queue.cc:1035] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:41.286075 31470 consensus_queue.cc:1035] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:44:41.287945 31476 mvcc.cc:205] Tried to move back new op lower bound from 7315450598542553088 to 7315450598446268416. Current Snapshot: MvccSnapshot[applied={T|T < 7315450598542553088}]
I20260806 06:44:41.288069 31475 mvcc.cc:205] Tried to move back new op lower bound from 7315450598542553088 to 7315450598446268416. Current Snapshot: MvccSnapshot[applied={T|T < 7315450598542553088 or (T in {7315450598542553088})}]
I20260806 06:44:41.288940 31478 mvcc.cc:205] Tried to move back new op lower bound from 7315450598542553088 to 7315450598446268416. Current Snapshot: MvccSnapshot[applied={T|T < 7315450598542553088 or (T in {7315450598542553088})}]
I20260806 06:44:41.290338 30977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:41.290442 30977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:44:41.291181 30977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Sending DeleteTablet for 3 replicas of tablet b57bc425e11c48a5a7352518b523c5ef
I20260806 06:44:41.291540 31387 tablet_service.cc:1514] Processing DeleteTablet for tablet b57bc425e11c48a5a7352518b523c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:41 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:51034
I20260806 06:44:41.291543 31248 tablet_service.cc:1514] Processing DeleteTablet for tablet b57bc425e11c48a5a7352518b523c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:41 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:56956
I20260806 06:44:41.291600 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31017
I20260806 06:44:41.291729 31104 tablet_service.cc:1514] Processing DeleteTablet for tablet b57bc425e11c48a5a7352518b523c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:44:41 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:48934
I20260806 06:44:41.292712 31487 tablet_replica.cc:330] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: stopping tablet replica
I20260806 06:44:41.292819 31487 raft_consensus.cc:2229] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:44:41.292927 31487 raft_consensus.cc:2258] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:41.293357 31487 ts_tablet_manager.cc:1905] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:41.293905 31486 tablet_replica.cc:330] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: stopping tablet replica
I20260806 06:44:41.293991 31486 raft_consensus.cc:2229] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:44:41.294091 31486 raft_consensus.cc:2258] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:44:41.294523 31486 ts_tablet_manager.cc:1905] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:44:41.295231 31487 ts_tablet_manager.cc:1918] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:41.295295 31487 log.cc:1196] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/b57bc425e11c48a5a7352518b523c5ef
I20260806 06:44:41.295588 31487 ts_tablet_manager.cc:1939] T b57bc425e11c48a5a7352518b523c5ef P c3748c6bf8e64b97b8145df3b961ece6: Deleting consensus metadata
I20260806 06:44:41.296293 31486 ts_tablet_manager.cc:1918] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:44:41.296339 31486 log.cc:1196] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/b57bc425e11c48a5a7352518b523c5ef
I20260806 06:44:41.296532 31486 ts_tablet_manager.cc:1939] T b57bc425e11c48a5a7352518b523c5ef P 2d8a9401f1a2475e879c366eb1e9ed15: Deleting consensus metadata
I20260806 06:44:41.297204 30963 catalog_manager.cc:4841] TS 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): tablet b57bc425e11c48a5a7352518b523c5ef (table test-table [id=188e325d8a2a4481b14f460f7d45a881]) successfully deleted
W20260806 06:44:41.303000 30960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:45499 (error 108)
I20260806 06:44:41.303509 30964 catalog_manager.cc:4841] TS c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): tablet b57bc425e11c48a5a7352518b523c5ef (table test-table [id=188e325d8a2a4481b14f460f7d45a881]) successfully deleted
I20260806 06:44:41.304054 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:44:41.303914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:45499 (local address 127.0.0.1:48936)
0806 06:44:41.303968 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:44:41.304022 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:45499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:44:41.304056 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31176
W20260806 06:44:41.304177 30960 catalog_manager.cc:4584] TS ae923b4850114eb1a39eb5108fab624b (127.14.51.65:45499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b57bc425e11c48a5a7352518b523c5ef: Network error: Client connection negotiation failed: client connection to 127.14.51.65:45499: connect: Connection refused (error 111)
I20260806 06:44:41.311789 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31321
I20260806 06:44:41.318272 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 30943
I20260806 06:45:01.325796 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--webserver_interface=127.14.51.126
--webserver_port=45901
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:01.406085 31489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:01.406267 31489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:01.406289 31489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:01.407523 31489 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:45:01.407563 31489 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:45:01.407577 31489 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:45:01.407588 31489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:01.407622 31489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:01.407649 31489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:01.407666 31489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:01.407689 31489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:01.409108 31489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=45901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:01.409376 31489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:01.409610 31489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:01.414345 31498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:01.414345 31495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.414443 31489 server_base.cc:1034] running on GCE node
W20260806 06:45:01.414378 31494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.414805 31489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:01.415064 31489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:01.416224 31489 hybrid_clock.cc:648] HybridClock initialized: now 1785998701416223 us; error 58 us; skew 500 ppm
Aug 06 06:45:01 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:01.418583 31489 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:01.418988 31489 webserver.cc:460] Webserver started at http://127.14.51.126:45901/ using document root <none> and password file <none>
I20260806 06:45:01.419263 31489 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:01.419332 31489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:01.420667 31489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.421344 31505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.421535 31489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.421631 31489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "83af52de4a7d4ebdbdac49346da8133b"
format_stamp: "Formatted at 2026-08-06 06:44:19 on dist-test-slave-1xrh"
server_key: "86d1b698947bfc2f014ba9aee3972a55"
server_key_iv: "a8cb8220ba0bbc1ffd37359c98402cff"
server_key_version: "encryptionkey@0"
I20260806 06:45:01.421943 31489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:01.429411 31489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:01.431325 31489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:01.431512 31489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:01.435755 31489 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:34719
I20260806 06:45:01.435788 31560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:34719 every 8 connection(s)
I20260806 06:45:01.436169 31489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:01.437525 31561 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:45:01.438076 31561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap starting.
I20260806 06:45:01.441073 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31489
I20260806 06:45:01.441366 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:45499
--local_ip_for_outbound_sockets=127.14.51.65
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=33211
--webserver_interface=127.14.51.65
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:01.446832 31561 log.cc:824] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:01.449005 31561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:45:01.449303 31561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Bootstrap complete.
I20260806 06:45:01.451530 31561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:45:01.451756 31561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Initialized, Role: FOLLOWER
I20260806 06:45:01.451918 31561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:45:01.451977 31561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:01.452044 31561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:01.452093 31561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:45:01.453064 31561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:45:01.453172 31561 leader_election.cc:302] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83af52de4a7d4ebdbdac49346da8133b; no voters:
I20260806 06:45:01.453367 31561 leader_election.cc:288] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:45:01.453400 31565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:45:01.453519 31565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [term 3 LEADER]: Becoming Leader. State: Replica: 83af52de4a7d4ebdbdac49346da8133b, State: Running, Role: LEADER
I20260806 06:45:01.453603 31565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } }
I20260806 06:45:01.453636 31561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:01.453873 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83af52de4a7d4ebdbdac49346da8133b. Latest consensus state: current_term: 3 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:45:01.453950 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:01.453876 31566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "83af52de4a7d4ebdbdac49346da8133b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83af52de4a7d4ebdbdac49346da8133b" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34719 } } }
I20260806 06:45:01.454133 31566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:01.454217 31569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:01.455085 31569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:01.455227 31569 catalog_manager.cc:1259] Loaded cluster ID: 8c76e8d37fe04fd790f2dbb1f76e657f
I20260806 06:45:01.455533 31569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:01.456326 31569 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:01.456524 31569 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Loaded TSK: 0
I20260806 06:45:01.456593 31569 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83af52de4a7d4ebdbdac49346da8133b: Loaded TSK: 1
F20260806 06:45:01.456804 31569 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f94dd9489ad google::LogMessage::Fail() at ??:0
@ 0x7f94dd94b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f94dd9484de google::LogMessage::Flush() at ??:0
@ 0x7f94dd949a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f94e00b5104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f94e00b2d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f94e58a21cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f94e58a248e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f94e58a2594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f94e5868c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f94e5872ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f94df8160d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f94df80b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f94e066f6db start_thread at ??:0
@ 0x7f94dc44661f clone at ??:0
W20260806 06:45:01.541795 31563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:01.541968 31563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:01.541989 31563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:01.543071 31563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:01.543111 31563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:01.543210 31563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:01.544672 31563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:45499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=33211
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:01.544886 31563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:01.545130 31563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:01.551172 31605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:01.551222 31603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:01.551225 31602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.551522 31563 server_base.cc:1034] running on GCE node
I20260806 06:45:01.551729 31563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:01.551975 31563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:01.553138 31563 hybrid_clock.cc:648] HybridClock initialized: now 1785998701553129 us; error 26 us; skew 500 ppm
Aug 06 06:45:01 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:01.555611 31563 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:45:01.556058 31563 webserver.cc:460] Webserver started at http://127.14.51.65:33211/ using document root <none> and password file <none>
I20260806 06:45:01.556317 31563 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:01.556362 31563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:01.557677 31563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.558331 31614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.558534 31563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.558600 31563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "ae923b4850114eb1a39eb5108fab624b"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "e4c31f8a5c0600cc1bd4064b7b9f09c8"
server_key_iv: "32043d8e9bf35293249bf11dade73aac"
server_key_version: "encryptionkey@0"
I20260806 06:45:01.558930 31563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:01.591080 31563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:01.593283 31563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:01.593430 31563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:01.593713 31563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:01.594254 31621 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:45:01.596288 31563 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:45:01.596349 31563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:45:01.596393 31563 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:45:01.597003 31563 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:45:01.597062 31563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:45:01.597129 31621 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Bootstrap starting.
I20260806 06:45:01.604274 31563 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:45499
I20260806 06:45:01.604398 31734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:45499 every 8 connection(s)
I20260806 06:45:01.604681 31563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:01.607759 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31563
I20260806 06:45:01.608058 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:33899
--local_ip_for_outbound_sockets=127.14.51.66
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=34563
--webserver_interface=127.14.51.66
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:01.608656 31621 log.cc:824] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:01.611334 31621 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:45:01.611649 31621 tablet_bootstrap.cc:492] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Bootstrap complete.
I20260806 06:45:01.612851 31621 ts_tablet_manager.cc:1397] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260806 06:45:01.614265 31621 raft_consensus.cc:348] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:01.614585 31621 raft_consensus.cc:729] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae923b4850114eb1a39eb5108fab624b, State: Initialized, Role: FOLLOWER
I20260806 06:45:01.614755 31621 consensus_queue.cc:260] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:01.615013 31621 ts_tablet_manager.cc:1428] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260806 06:45:01.645962 31737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:01.605395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:56897)
0806 06:45:01.605600 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.605632 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:01.606084 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:01.645818 (+ 39734us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":119,"threads_started":1}
W20260806 06:45:01.646229 31735 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:34719 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:34719: Client connection negotiation failed: client connection to 127.14.51.126:34719: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:45:01.646557 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.646435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:33755)
0806 06:45:01.646489 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.646530 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:45:01.646960 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.646871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:39551)
0806 06:45:01.646920 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.646944 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:45:01.647142 31735 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:45:01.692659 31739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:01.692852 31739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:01.692874 31739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:01.694077 31739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:01.694114 31739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:01.694161 31739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:01.695778 31739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:33899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=34563
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:01.696022 31739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:01.696236 31739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:01.701050 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:01.701088 31746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.701150 31739 server_base.cc:1034] running on GCE node
W20260806 06:45:01.701051 31745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.701460 31739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:01.701707 31739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:01.702867 31739 hybrid_clock.cc:648] HybridClock initialized: now 1785998701702829 us; error 72 us; skew 500 ppm
Aug 06 06:45:01 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:01.705307 31739 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:45:01.705700 31739 webserver.cc:460] Webserver started at http://127.14.51.66:34563/ using document root <none> and password file <none>
I20260806 06:45:01.705915 31739 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:01.705979 31739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:01.707255 31739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.707810 31755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.707966 31739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:01.708060 31739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "c3748c6bf8e64b97b8145df3b961ece6"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "41d22edd33b94b5482e766cf474e688d"
server_key_iv: "a1fb2bf82eeac2434e83459781f527d5"
server_key_version: "encryptionkey@0"
I20260806 06:45:01.708366 31739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:01.717823 31739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:01.720038 31739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:01.720191 31739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:01.720400 31739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:01.720695 31739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:01.720752 31739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.720793 31739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:01.720820 31739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.726764 31739 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:33899
I20260806 06:45:01.726840 31875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:33899 every 8 connection(s)
I20260806 06:45:01.727250 31739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:01.728379 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.728023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:37289)
0806 06:45:01.728185 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.728248 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260806 06:45:01.728811 31876 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:34719 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:34719: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
I20260806 06:45:01.729077 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.728986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:51611)
0806 06:45:01.729036 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.729061 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:45:01.729332 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.729263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:53821)
0806 06:45:01.729299 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.729319 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:45:01.729432 31876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:45:01.733932 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31739
I20260806 06:45:01.734218 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:37875
--local_ip_for_outbound_sockets=127.14.51.67
--tserver_master_addrs=127.14.51.126:34719
--webserver_port=34639
--webserver_interface=127.14.51.67
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:01.813130 31879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:01.813340 31879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:01.813377 31879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:01.814463 31879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:01.814534 31879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:01.814579 31879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:01.816104 31879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:37875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=34639
--tserver_master_addrs=127.14.51.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:01.816344 31879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:01.816555 31879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:01.821450 31885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.821549 31879 server_base.cc:1034] running on GCE node
W20260806 06:45:01.821457 31887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:01.821462 31884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:01.821830 31879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:01.822062 31879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:01.823180 31879 hybrid_clock.cc:648] HybridClock initialized: now 1785998701823160 us; error 40 us; skew 500 ppm
Aug 06 06:45:01 dist-test-slave-1xrh krb5kdc[30383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:01.825484 31879 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:45:01.825836 31879 webserver.cc:460] Webserver started at http://127.14.51.67:34639/ using document root <none> and password file <none>
I20260806 06:45:01.825992 31879 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:01.826042 31879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:01.827277 31879 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:01.827803 31894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.827968 31879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.828068 31879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "2d8a9401f1a2475e879c366eb1e9ed15"
format_stamp: "Formatted at 2026-08-06 06:44:20 on dist-test-slave-1xrh"
server_key: "53a57cfa67e6334af5e69751be2b6ef4"
server_key_iv: "e8364eb4fc596a70571be22a38ab4144"
server_key_version: "encryptionkey@0"
I20260806 06:45:01.828406 31879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:01.837667 31879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:01.839663 31879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:01.839802 31879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:01.839998 31879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:01.840269 31879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:01.840317 31879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.840346 31879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:01.840370 31879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:01.845986 31879 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37875
I20260806 06:45:01.846031 32011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37875 every 8 connection(s)
I20260806 06:45:01.846314 31879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:01.847288 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.846901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:41019)
0806 06:45:01.847038 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.847098 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260806 06:45:01.847645 32012 heartbeater.cc:644] Failed to heartbeat to 127.14.51.126:34719 (0 consecutive failures): Network error: Failed to ping master at 127.14.51.126:34719: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
I20260806 06:45:01.847905 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.847815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:35801)
0806 06:45:01.847861 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.847885 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:45:01.848176 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:01.848096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:41425)
0806 06:45:01.848136 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:01.848157 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:45:01.848280 32012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:45:01.849001 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 31879
I20260806 06:45:02.647701 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:02.647469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:46063)
0806 06:45:02.647607 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:02.647641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260806 06:45:02.729990 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:02.729699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:56395)
0806 06:45:02.729891 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:02.729925 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":53,"threads_started":1}
I20260806 06:45:02.849095 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:02.848610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:48267)
0806 06:45:02.848853 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:02.849027 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":52,"threads_started":1}
I20260806 06:45:03.011104 32019 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:03.011278 32019 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:03.011601 32019 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899), 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
Aug 06 06:45:03 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.66@KRBTEST.COM
Aug 06 06:45:03 dist-test-slave-1xrh krb5kdc[30383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998701, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.67@KRBTEST.COM
I20260806 06:45:03.021538 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.011777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33899 (local address 127.14.51.65:55655)
0806 06:45:03.011915 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:03.011925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:03.011979 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:03.012716 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:03.012719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:03.013122 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:03.013296 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:03.013301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.014393 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.014394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:03.015223 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:03.015226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.015832 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:03.016003 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:03.016627 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:03.016634 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:03.017952 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:03.020066 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:03.020075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:03.020296 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:03.020555 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:03.020557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:03.020666 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:03.021044 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:03.021051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:03.021176 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 06:45:03.021276 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:45:03.021426 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":70,"thread_start_us":79,"threads_started":1}
I20260806 06:45:03.021574 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.011897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:55655 (local address 127.14.51.66:33899)
0806 06:45:03.012073 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:03.012082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:03.012111 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:45:03.012518 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:03.012520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:03.012552 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:03.012691 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:03.013412 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.014258 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015325 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.015758 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015858 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:03.018058 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:03.018062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:03.018062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:03.018073 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:03.019977 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020391 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:03.020394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:03.020394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:03.020481 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020737 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:03.020738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:03.020738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:03.020959 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:03.021051 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:03.021339 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 06:45:03.021469 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":51,"threads_started":1}
I20260806 06:45:03.021677 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.011879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:54049 (local address 127.14.51.67:37875)
0806 06:45:03.012077 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:03.012084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:03.012109 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:45:03.012505 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:03.012513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:03.012552 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:03.012688 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:03.013404 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.014230 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015294 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.015740 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015859 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:03.018507 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:03.018510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:03.018511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:03.018523 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:03.020320 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020680 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:03.020683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:03.020683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:03.020768 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020968 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:03.020969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:03.020970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:03.021184 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:03.021266 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:03.021448 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:45:03.021571 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":60,"threads_started":1}
I20260806 06:45:03.021639 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.011759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37875 (local address 127.14.51.65:54049)
0806 06:45:03.011906 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:03.011919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:03.011978 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:03.012716 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:03.012719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:03.013123 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:03.013296 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:03.013301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.014330 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.014331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:03.015215 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:03.015218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:03.015918 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:03.015919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:03.016002 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:03.016573 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:03.016581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:03.018414 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:03.020422 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:03.020424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:03.020600 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:03.020835 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:03.020836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:03.020836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:03.020896 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:03.021262 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:03.021263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:03.021304 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:45:03.021392 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:45:03.021487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2}
W20260806 06:45:03.022660 31617 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
W20260806 06:45:03.022857 31618 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
I20260806 06:45:03.022921 31618 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae923b4850114eb1a39eb5108fab624b; no voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6
I20260806 06:45:03.023036 32019 raft_consensus.cc:2726] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:45:03.648344 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.648138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:39025)
0806 06:45:03.648264 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:03.648300 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260806 06:45:03.730729 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.730535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:44931)
0806 06:45:03.730664 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:03.730695 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260806 06:45:03.849705 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:03.849514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:43449)
0806 06:45:03.849634 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:03.849668 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260806 06:45:04.648947 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:04.648748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:59467)
0806 06:45:04.648870 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:04.648902 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260806 06:45:04.652355 32028 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:04.652426 32028 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:04.652606 32028 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899), 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
W20260806 06:45:04.653020 31618 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
W20260806 06:45:04.653082 31617 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
I20260806 06:45:04.653110 31618 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae923b4850114eb1a39eb5108fab624b; no voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6
I20260806 06:45:04.653177 32028 raft_consensus.cc:2726] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:45:04.731442 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:04.731255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:54029)
0806 06:45:04.731367 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:04.731404 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260806 06:45:04.850337 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:04.850132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:45437)
0806 06:45:04.850268 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:04.850300 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260806 06:45:05.649641 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:05.649414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:58643)
0806 06:45:05.649559 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:05.649595 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260806 06:45:05.732017 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:05.731839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:35073)
0806 06:45:05.731948 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:05.731981 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260806 06:45:05.850924 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:05.850709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:60681)
0806 06:45:05.850857 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:05.850889 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260806 06:45:06.650283 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:06.650073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:39487)
0806 06:45:06.650201 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:06.650236 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260806 06:45:06.732586 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:06.732389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:56029)
0806 06:45:06.732518 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:06.732551 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 06:45:06.784384 32036 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:06.784487 32036 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:06.784693 32036 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899), 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
W20260806 06:45:06.785032 31618 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
W20260806 06:45:06.785116 31617 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
I20260806 06:45:06.785198 31617 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae923b4850114eb1a39eb5108fab624b; no voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6
I20260806 06:45:06.785292 32036 raft_consensus.cc:2726] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:45:06.851517 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:06.851318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:53563)
0806 06:45:06.851436 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:06.851470 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260806 06:45:07.651072 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:07.650827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:37929)
0806 06:45:07.650989 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:07.651026 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260806 06:45:07.733117 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:07.732950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:34727)
0806 06:45:07.733052 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:07.733082 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260806 06:45:07.852099 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:07.851912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:33693)
0806 06:45:07.852024 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:07.852059 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260806 06:45:08.651742 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:08.651526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:49051)
0806 06:45:08.651657 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:08.651694 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260806 06:45:08.733665 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:08.733491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:55549)
0806 06:45:08.733598 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:08.733629 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260806 06:45:08.852818 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:08.852506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:51887)
0806 06:45:08.852730 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:08.852775 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":58,"threads_started":1}
I20260806 06:45:09.505722 32044 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:09.505806 32044 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:09.505960 32044 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899), 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
W20260806 06:45:09.506325 31617 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
W20260806 06:45:09.506421 31618 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
I20260806 06:45:09.506501 31618 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae923b4850114eb1a39eb5108fab624b; no voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6
I20260806 06:45:09.506582 32044 raft_consensus.cc:2726] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:45:09.652427 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:09.652212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:54697)
0806 06:45:09.652349 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:09.652384 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260806 06:45:09.734253 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:09.734059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:45065)
0806 06:45:09.734167 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:09.734198 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260806 06:45:09.853477 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:09.853284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:49227)
0806 06:45:09.853395 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:09.853425 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260806 06:45:10.653129 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:10.652923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:50675)
0806 06:45:10.653049 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:10.653085 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260806 06:45:10.734891 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:10.734680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:58287)
0806 06:45:10.734821 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:10.734854 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260806 06:45:10.854158 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:10.853912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:59823)
0806 06:45:10.854069 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:10.854105 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260806 06:45:11.653852 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:11.653635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:47859)
0806 06:45:11.653773 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:11.653806 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260806 06:45:11.735500 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:11.735310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:46631)
0806 06:45:11.735429 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:11.735459 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260806 06:45:11.854893 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:11.854664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:48895)
0806 06:45:11.854810 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:11.854850 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260806 06:45:12.654512 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:12.654281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:42153)
0806 06:45:12.654422 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:12.654455 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260806 06:45:12.736099 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:12.735911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:55593)
0806 06:45:12.736022 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:12.736056 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260806 06:45:12.855538 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:12.855327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:35129)
0806 06:45:12.855455 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:12.855490 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260806 06:45:13.655308 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:13.655077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:55869)
0806 06:45:13.655226 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:13.655261 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260806 06:45:13.736716 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:13.736517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:34571)
0806 06:45:13.736639 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:13.736673 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260806 06:45:13.856185 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:13.855965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:32809)
0806 06:45:13.856109 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:13.856144 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260806 06:45:14.655973 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:14.655767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:57447)
0806 06:45:14.655894 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:14.655928 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260806 06:45:14.737378 32061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:14.737180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:59443)
0806 06:45:14.737302 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:14.737334 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 06:45:14.801868 32062 raft_consensus.cc:482] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:14.801965 32062 raft_consensus.cc:504] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3748c6bf8e64b97b8145df3b961ece6" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33899 } } peers { permanent_uuid: "2d8a9401f1a2475e879c366eb1e9ed15" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37875 } } peers { permanent_uuid: "ae923b4850114eb1a39eb5108fab624b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 45499 } }
I20260806 06:45:14.802198 32062 leader_election.cc:288] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899), 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875)
W20260806 06:45:14.802620 31618 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c3748c6bf8e64b97b8145df3b961ece6 (127.14.51.66:33899): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
W20260806 06:45:14.802721 31617 leader_election.cc:341] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2d8a9401f1a2475e879c366eb1e9ed15 (127.14.51.67:37875): Not found: Tablet not found: b57bc425e11c48a5a7352518b523c5ef
I20260806 06:45:14.802816 31617 leader_election.cc:302] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae923b4850114eb1a39eb5108fab624b; no voters: 2d8a9401f1a2475e879c366eb1e9ed15, c3748c6bf8e64b97b8145df3b961ece6
I20260806 06:45:14.802923 32062 raft_consensus.cc:2726] T b57bc425e11c48a5a7352518b523c5ef P ae923b4850114eb1a39eb5108fab624b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:45:14.856868 32063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:14.856676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:39983)
0806 06:45:14.856790 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:14.856825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260806 06:45:15.656596 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:15.656362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:40353)
0806 06:45:15.656511 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:15.656549 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260806 06:45:15.738014 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:15.737801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:54417)
0806 06:45:15.737933 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:15.737966 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 06:45:15.857542 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:15.857309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:44985)
0806 06:45:15.857455 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:15.857491 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260806 06:45:16.657459 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:16.657224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.65:40773)
0806 06:45:16.657381 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:16.657414 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260806 06:45:16.738899 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:16.738680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.66:34777)
0806 06:45:16.738809 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:16.738841 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260806 06:45:16.857666 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31563
I20260806 06:45:16.859853 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:16.859294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34719 (local address 127.14.51.67:44151)
0806 06:45:16.859724 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:16.859762 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.126:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":153,"threads_started":1}
I20260806 06:45:16.863263 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31739
I20260806 06:45:16.868306 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 31879
2026-08-06T06:45:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58307 ms)
[----------] 30 tests from SecurityITest (234177 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:16 dist-test-slave-1xrh krb5kdc[32072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:16 dist-test-slave-1xrh krb5kdc[32072](info): set up 2 sockets
Aug 06 06:45:16 dist-test-slave-1xrh krb5kdc[32072](info): commencing operation
krb5kdc: starting...
W20260806 06:45:17.934676 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:18.049814 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44643
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:44643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:18.126562 32085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:18.126812 32085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:18.126848 32085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:18.127946 32085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:18.128023 32085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:18.128047 32085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:18.128067 32085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:18.128094 32085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:18.129455 32085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:44643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:18.129721 32085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:18.129944 32085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:18.134828 32091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.134882 32090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.134923 32085 server_base.cc:1034] running on GCE node
W20260806 06:45:18.135016 32093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.135279 32085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:18.135584 32085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:18.136762 32085 hybrid_clock.cc:648] HybridClock initialized: now 1785998718136744 us; error 49 us; skew 500 ppm
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:18.139346 32085 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:18.139701 32085 webserver.cc:460] Webserver started at http://127.14.51.126:37503/ using document root <none> and password file <none>
I20260806 06:45:18.139866 32085 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:18.139906 32085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:18.139988 32085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:18.140884 32085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "72a5ad90deb449dda74c2f2101445610"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "63510f47da9ec4bea18e094f7fc57996"
server_key_iv: "6748504ce7da05bb18e50c63ce68b781"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.141176 32085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "72a5ad90deb449dda74c2f2101445610"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "63510f47da9ec4bea18e094f7fc57996"
server_key_iv: "6748504ce7da05bb18e50c63ce68b781"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.142409 32085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:18.143088 32100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.143293 32085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:18.143375 32085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "72a5ad90deb449dda74c2f2101445610"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "63510f47da9ec4bea18e094f7fc57996"
server_key_iv: "6748504ce7da05bb18e50c63ce68b781"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.143471 32085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:18.175438 32085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:18.177313 32085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:18.177474 32085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:18.181360 32085 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:44643
I20260806 06:45:18.181420 32156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:44643 every 8 connection(s)
I20260806 06:45:18.181768 32085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:18.182606 32157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:18.184608 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32085
I20260806 06:45:18.184719 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:18.184938 14541 external_mini_cluster.cc:1428] Setting key 497b256df0b4ee948ba4236555ef53bc
I20260806 06:45:18.184978 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: Bootstrap starting.
I20260806 06:45:18.185680 32157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:18.185982 32157 log.cc:824] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:18.186795 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: No bootstrap required, opened a new log
I20260806 06:45:18.188928 32157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } }
I20260806 06:45:18.189085 32157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:18.189113 32157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a5ad90deb449dda74c2f2101445610, State: Initialized, Role: FOLLOWER
I20260806 06:45:18.189218 32157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } }
I20260806 06:45:18.189288 32157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:18.189342 32157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:18.189409 32157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:18.189960 32157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } }
I20260806 06:45:18.190073 32157 leader_election.cc:302] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a5ad90deb449dda74c2f2101445610; no voters:
I20260806 06:45:18.190719 32157 leader_election.cc:288] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:18.190754 32162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:18.191013 32162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [term 1 LEADER]: Becoming Leader. State: Replica: 72a5ad90deb449dda74c2f2101445610, State: Running, Role: LEADER
I20260806 06:45:18.191023 32157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:18.191195 32162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } }
I20260806 06:45:18.191493 32164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a5ad90deb449dda74c2f2101445610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } } }
I20260806 06:45:18.191556 32164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:18.191795 32167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:18.192296 32167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:18.192967 32163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a5ad90deb449dda74c2f2101445610. Latest consensus state: current_term: 1 leader_uuid: "72a5ad90deb449dda74c2f2101445610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a5ad90deb449dda74c2f2101445610" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44643 } } }
I20260806 06:45:18.193130 32163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:18.193794 32181 mvcc.cc:205] Tried to move back new op lower bound from 7315450749719326720 to 7315450749711441920. Current Snapshot: MvccSnapshot[applied={T|T < 7315450749719326720}]
I20260806 06:45:18.194227 32167 catalog_manager.cc:1347] Generated new cluster ID: 45d227f1b1fd48c785c1a7945c388a06
I20260806 06:45:18.194279 32167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:18.196998 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.185522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.14.51.126:44643)
0806 06:45:18.185692 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.185700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.185746 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:45:18.186295 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.186305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.186385 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.186536 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:18.187333 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.188250 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.189362 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.189834 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.189914 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.193105 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:18.193109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:18.193110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:18.193121 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:18.195350 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.195786 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.195788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.195789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.195893 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.196116 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.196117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.196117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.196376 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:18.196509 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.196675 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.196809 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260806 06:45:18.210553 32167 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:18.211061 32167 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:18.213703 32167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: Generated new TSK 0
I20260806 06:45:18.213873 32167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:45:18.242184 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:18.322317 32186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:18.322510 32186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:18.322532 32186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:18.323691 32186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:18.323734 32186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:18.323792 32186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:18.325279 32186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:18.325536 32186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:18.325775 32186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:18.331079 32192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.331087 32191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.331113 32194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.331308 32186 server_base.cc:1034] running on GCE node
I20260806 06:45:18.331527 32186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:18.331724 32186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:18.332872 32186 hybrid_clock.cc:648] HybridClock initialized: now 1785998718332897 us; error 63 us; skew 500 ppm
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:18.335345 32186 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:45:18.335788 32186 webserver.cc:460] Webserver started at http://127.14.51.65:42941/ using document root <none> and password file <none>
I20260806 06:45:18.336040 32186 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:18.336117 32186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:18.336274 32186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:18.337474 32186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "89aeb17087ff432792cf9cb7c3c08ee4"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "0ba7d475047a2707ee6044e26cbaf36d"
server_key_iv: "b5d2a94a5393c5935ec5ee4c6c88fb87"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.337921 32186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "89aeb17087ff432792cf9cb7c3c08ee4"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "0ba7d475047a2707ee6044e26cbaf36d"
server_key_iv: "b5d2a94a5393c5935ec5ee4c6c88fb87"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.339780 32186 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:45:18.340770 32203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.341017 32186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:18.341113 32186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "89aeb17087ff432792cf9cb7c3c08ee4"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "0ba7d475047a2707ee6044e26cbaf36d"
server_key_iv: "b5d2a94a5393c5935ec5ee4c6c88fb87"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.341192 32186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:18.353659 32186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:18.355600 32186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:18.355705 32186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:18.355866 32186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:18.356169 32186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:18.356202 32186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.356223 32186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:18.356236 32186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.361845 32186 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:36859
I20260806 06:45:18.361930 32326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:36859 every 8 connection(s)
I20260806 06:45:18.362241 32186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:18.366871 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32186
I20260806 06:45:18.366987 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:18.367252 14541 external_mini_cluster.cc:1428] Setting key 218dfe5f2e500d2dc44a6ec84690d947
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:18.372385 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.362973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44643 (local address 127.14.51.65:43081)
0806 06:45:18.363185 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.363227 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.363720 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.363992 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.363996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.364549 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:18.364725 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.364729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.365440 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.365441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.366202 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.366204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.366609 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.366610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.366796 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.367364 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:18.367373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:18.368927 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:18.370946 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.370952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.370953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.371243 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.371452 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.371453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.371453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.371546 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.371904 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:18.371911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:18.372045 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.372123 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.372235 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260806 06:45:18.372403 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.363009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43081 (local address 127.14.51.126:44643)
0806 06:45:18.363053 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.363055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.363320 (+ 265us) server_negotiation.cc:407] Connection header received
0806 06:45:18.363853 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.363855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.363867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.363923 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:18.364837 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.365344 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.366297 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.366521 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.366602 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.369098 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:18.369100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:18.369100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:18.369108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:18.370871 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.371317 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.371319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.371319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.371383 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.371633 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.371634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.371634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.371823 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:18.371898 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.372201 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.372291 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:45:18.372961 32327 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44643
I20260806 06:45:18.373051 32327 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:18.373230 32327 heartbeater.cc:511] Master 127.14.51.126:44643 requested a full tablet report, sending...
I20260806 06:45:18.373700 32118 ts_manager.cc:194] Registered new tserver with Master: 89aeb17087ff432792cf9cb7c3c08ee4 (127.14.51.65:36859)
I20260806 06:45:18.374259 32118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:43081
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:45:18.394693 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:18.473184 32332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:18.473436 32332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:18.473474 32332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:18.474617 32332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:18.474692 32332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:18.474736 32332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:18.476143 32332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:18.476413 32332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:18.476648 32332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:18.481515 32338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.481539 32340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.481729 32332 server_base.cc:1034] running on GCE node
W20260806 06:45:18.481524 32337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.481974 32332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:18.482196 32332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:18.483345 32332 hybrid_clock.cc:648] HybridClock initialized: now 1785998718483320 us; error 41 us; skew 500 ppm
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:18.485764 32332 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:45:18.486188 32332 webserver.cc:460] Webserver started at http://127.14.51.66:42663/ using document root <none> and password file <none>
I20260806 06:45:18.486394 32332 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:18.486435 32332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:18.486585 32332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:18.487665 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "66e34c0291f94a3fbb0520b8fd91b0b8"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "5da1a6a67ad7a3d02d44c3a2f37a819a"
server_key_iv: "8a10890bd6c3d5e689e27f1c45151cde"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.488009 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "66e34c0291f94a3fbb0520b8fd91b0b8"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "5da1a6a67ad7a3d02d44c3a2f37a819a"
server_key_iv: "8a10890bd6c3d5e689e27f1c45151cde"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.489279 32332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:18.489966 32349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.490162 32332 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:18.490247 32332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "66e34c0291f94a3fbb0520b8fd91b0b8"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "5da1a6a67ad7a3d02d44c3a2f37a819a"
server_key_iv: "8a10890bd6c3d5e689e27f1c45151cde"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.490298 32332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:18.497710 32332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:18.499447 32332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:18.499557 32332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:18.499703 32332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:18.499985 32332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:18.500021 32332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.500041 32332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:18.500092 32332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.505498 32332 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:38311
I20260806 06:45:18.505584 32471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:38311 every 8 connection(s)
I20260806 06:45:18.505856 32332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:18.508756 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32332
I20260806 06:45:18.508867 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:18.509157 14541 external_mini_cluster.cc:1428] Setting key 778b8c8c50fd89fa076ee988d950abb0
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:18.516094 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.506496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44643 (local address 127.14.51.66:44989)
0806 06:45:18.506679 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.506710 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.507219 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.507437 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.507440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.507958 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:18.508148 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.508152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.508972 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.508974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.509999 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.510002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.510382 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.510383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.510540 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.510983 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:18.510992 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:18.512807 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:18.514771 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.514776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.514778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.514993 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.515237 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.515237 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.515238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.515333 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.515649 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:18.515654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:18.515755 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.515832 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.515942 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260806 06:45:18.516094 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.506527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44989 (local address 127.14.51.126:44643)
0806 06:45:18.506616 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.506619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.506799 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:45:18.507319 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.507320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.507331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.507377 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:18.508224 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.508851 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.510089 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.510284 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.510339 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.512912 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:18.512914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:18.512914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:18.512923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:18.514683 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.515081 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.515082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.515083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.515164 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.515412 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.515413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.515413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.515578 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:18.515656 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.515893 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.515986 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 06:45:18.516667 32472 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44643
I20260806 06:45:18.516772 32472 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:18.516953 32472 heartbeater.cc:511] Master 127.14.51.126:44643 requested a full tablet report, sending...
I20260806 06:45:18.517311 32118 ts_manager.cc:194] Registered new tserver with Master: 66e34c0291f94a3fbb0520b8fd91b0b8 (127.14.51.66:38311)
I20260806 06:45:18.517701 32118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:44989
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:45:18.537704 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:18.615767 32478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:18.615986 32478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:18.616032 32478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:18.617122 32478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:18.617198 32478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:18.617264 32478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:18.618719 32478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:18.618993 32478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:18.619218 32478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:18.623970 32484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.623978 32483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:18.624002 32486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:18.624214 32478 server_base.cc:1034] running on GCE node
I20260806 06:45:18.624508 32478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:18.624751 32478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:18.625916 32478 hybrid_clock.cc:648] HybridClock initialized: now 1785998718625911 us; error 40 us; skew 500 ppm
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:18.628336 32478 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:45:18.628774 32478 webserver.cc:460] Webserver started at http://127.14.51.67:33977/ using document root <none> and password file <none>
I20260806 06:45:18.628954 32478 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:18.628991 32478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:18.629110 32478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:18.629985 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "846c9f98db9247e68238398aee53cfc5"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "333ec094b47e75c6381cdee13862e2f6"
server_key_iv: "b422d3d0271a4fd7cb132466bb68bf8e"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.630265 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "846c9f98db9247e68238398aee53cfc5"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "333ec094b47e75c6381cdee13862e2f6"
server_key_iv: "b422d3d0271a4fd7cb132466bb68bf8e"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.631390 32478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:18.632059 32493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.632241 32478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:18.632330 32478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "846c9f98db9247e68238398aee53cfc5"
format_stamp: "Formatted at 2026-08-06 06:45:18 on dist-test-slave-1xrh"
server_key: "333ec094b47e75c6381cdee13862e2f6"
server_key_iv: "b422d3d0271a4fd7cb132466bb68bf8e"
server_key_version: "encryptionkey@0"
I20260806 06:45:18.632401 32478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:18.657747 32478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:18.660195 32478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:18.660365 32478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:18.660631 32478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:18.660951 32478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:18.660991 32478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.661039 32478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:18.661067 32478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:18.666736 32478 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:33283
I20260806 06:45:18.666817 32614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:33283 every 8 connection(s)
I20260806 06:45:18.667181 32478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:18.672497 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32478
I20260806 06:45:18.672624 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:18.672834 14541 external_mini_cluster.cc:1428] Setting key 1914eabe9e545fec1236f4cb1248c8dc
Aug 06 06:45:18 dist-test-slave-1xrh krb5kdc[32072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998718, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:18.676324 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.667955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:38261 (local address 127.14.51.126:44643)
0806 06:45:18.668052 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.668054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.668212 (+ 158us) server_negotiation.cc:407] Connection header received
0806 06:45:18.668677 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.668678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.668688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.668754 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:18.669635 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.670107 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.671199 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.671390 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.671444 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.673731 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:18.673733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:18.673733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:18.673741 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:18.675224 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.675587 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.675587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.675588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.675636 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.675838 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.675839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.675839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.675934 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:18.675990 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.676205 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.676263 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:45:18.676492 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.667906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44643 (local address 127.14.51.67:38261)
0806 06:45:18.668091 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.668131 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.668597 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.668813 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.668816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.669379 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:18.669566 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.669570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.670188 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.670189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.671048 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.671050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.671462 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.671463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.671568 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.672001 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:18.672009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:18.673622 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:18.675306 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.675311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.675313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.675505 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.675694 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:18.675694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:18.675695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:18.675782 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:18.675987 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:18.675991 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:18.676096 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.676172 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.676280 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260806 06:45:18.676972 32615 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44643
I20260806 06:45:18.677047 32615 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:18.677171 32615 heartbeater.cc:511] Master 127.14.51.126:44643 requested a full tablet report, sending...
I20260806 06:45:18.677503 32118 ts_manager.cc:194] Registered new tserver with Master: 846c9f98db9247e68238398aee53cfc5 (127.14.51.67:33283)
I20260806 06:45:18.677845 32118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:38261
I20260806 06:45:18.684610 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:18.692021 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.685949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.14.51.126:44643)
0806 06:45:18.686033 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.686035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.686101 (+ 66us) server_negotiation.cc:407] Connection header received
0806 06:45:18.686172 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.686173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.686183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.686256 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:18.686821 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.687414 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.688141 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.688316 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.688447 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.689579 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:18.689580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:18.689581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:18.689587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:18.690661 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.691068 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.691069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.691069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.691179 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:18.691395 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:18.691396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:18.691396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:18.691536 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:18.691647 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.691806 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.691884 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:45:18.693671 32118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:18.695309 32118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:18.701545 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:36859 (local address 127.0.0.1:37280)
0806 06:45:18.697807 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.697815 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.697856 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.698168 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.698169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.698171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:18.698214 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.698215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699460 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.699461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.700385 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.700388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701238 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.701239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.701318 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.701330 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.701401 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.701448 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":41,"threads_started":1}
I20260806 06:45:18.701581 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.14.51.65:36859)
0806 06:45:18.697931 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.697936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.697946 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:18.697989 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.697991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.698005 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.698087 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:18.698287 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699370 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.700474 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701139 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.701212 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.701263 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.701443 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.701504 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260806 06:45:18.701735 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.14.51.67:33283)
0806 06:45:18.698049 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.698053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.698107 (+ 54us) server_negotiation.cc:407] Connection header received
0806 06:45:18.698188 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.698190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.698204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.698273 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:18.698472 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699208 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.700743 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701246 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.701310 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.701361 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.701586 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.701629 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260806 06:45:18.701790 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:33283 (local address 127.0.0.1:57782)
0806 06:45:18.698055 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.698065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.698118 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.698341 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.698341 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.698343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:18.698384 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.698385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699322 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.699323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.700660 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.700662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701393 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.701394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.701444 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.701447 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.701491 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.701524 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":80,"thread_start_us":50,"threads_started":1}
I20260806 06:45:18.702178 32623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:38311 (local address 127.0.0.1:57196)
0806 06:45:18.697523 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.697561 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.697657 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.697964 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.697966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.697968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:18.698032 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.698033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699403 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.699404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.700635 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.700637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701526 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.701527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.701590 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.701594 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.702064 (+ 470us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.702109 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":59,"thread_start_us":45,"threads_started":1}
I20260806 06:45:18.702261 32254 tablet_service.cc:1467] Processing CreateTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 (DEFAULT_TABLE table=test-table [id=9d85eaf568724c16972f7a473824f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:18.702391 32546 tablet_service.cc:1467] Processing CreateTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 (DEFAULT_TABLE table=test-table [id=9d85eaf568724c16972f7a473824f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:18.702625 32625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.697568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.14.51.66:38311)
0806 06:45:18.697758 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.697763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.697774 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:45:18.697815 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.697817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.697834 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.697951 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:18.698120 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.699304 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.700731 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.701427 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.702292 (+ 865us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.702352 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.702378 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.702536 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260806 06:45:18.702636 32254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d527d126f0bd40bc93932e14cdd1ff80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:18.702726 32546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d527d126f0bd40bc93932e14cdd1ff80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:18.703722 32401 tablet_service.cc:1467] Processing CreateTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 (DEFAULT_TABLE table=test-table [id=9d85eaf568724c16972f7a473824f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:18.704092 32401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d527d126f0bd40bc93932e14cdd1ff80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:18.705412 32630 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Bootstrap starting.
I20260806 06:45:18.705966 32631 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Bootstrap starting.
I20260806 06:45:18.706142 32630 tablet_bootstrap.cc:654] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:18.706635 32631 tablet_bootstrap.cc:654] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:18.706701 32630 log.cc:824] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:18.707230 32631 log.cc:824] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:18.707238 32633 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Bootstrap starting.
I20260806 06:45:18.707597 32630 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: No bootstrap required, opened a new log
I20260806 06:45:18.707712 32630 ts_tablet_manager.cc:1397] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:45:18.707868 32633 tablet_bootstrap.cc:654] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:18.708060 32631 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: No bootstrap required, opened a new log
I20260806 06:45:18.708146 32631 ts_tablet_manager.cc:1397] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:45:18.708393 32633 log.cc:824] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:18.709167 32633 tablet_bootstrap.cc:492] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: No bootstrap required, opened a new log
I20260806 06:45:18.709230 32633 ts_tablet_manager.cc:1397] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:18.709542 32630 raft_consensus.cc:348] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.709700 32630 raft_consensus.cc:374] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:18.709743 32630 raft_consensus.cc:729] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 846c9f98db9247e68238398aee53cfc5, State: Initialized, Role: FOLLOWER
I20260806 06:45:18.709853 32630 consensus_queue.cc:260] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.710150 32630 ts_tablet_manager.cc:1428] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:45:18.710045 32631 raft_consensus.cc:348] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.710191 32631 raft_consensus.cc:374] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:18.710222 32631 raft_consensus.cc:729] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89aeb17087ff432792cf9cb7c3c08ee4, State: Initialized, Role: FOLLOWER
I20260806 06:45:18.710285 32615 heartbeater.cc:503] Master 127.14.51.126:44643 was elected leader, sending a full tablet report...
I20260806 06:45:18.710346 32631 consensus_queue.cc:260] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.710598 32631 ts_tablet_manager.cc:1428] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:45:18.710615 32327 heartbeater.cc:503] Master 127.14.51.126:44643 was elected leader, sending a full tablet report...
I20260806 06:45:18.711218 32633 raft_consensus.cc:348] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.711380 32633 raft_consensus.cc:374] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:18.711434 32633 raft_consensus.cc:729] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e34c0291f94a3fbb0520b8fd91b0b8, State: Initialized, Role: FOLLOWER
I20260806 06:45:18.711534 32633 consensus_queue.cc:260] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.711796 32633 ts_tablet_manager.cc:1428] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:18.711855 32472 heartbeater.cc:503] Master 127.14.51.126:44643 was elected leader, sending a full tablet report...
W20260806 06:45:18.756443 32474 tablet.cc:2369] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:18.860580 32636 raft_consensus.cc:482] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:18.860710 32636 raft_consensus.cc:504] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.861023 32636 leader_election.cc:288] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e34c0291f94a3fbb0520b8fd91b0b8 (127.14.51.66:38311), 89aeb17087ff432792cf9cb7c3c08ee4 (127.14.51.65:36859)
W20260806 06:45:18.863049 32328 tablet.cc:2369] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:18.864204 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.861249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:38311 (local address 127.14.51.67:38829)
0806 06:45:18.861331 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.861364 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.861416 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.861653 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.861654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.861656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:18.861732 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.861734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.862386 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.862387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.863284 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.863287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.863924 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.863925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.864017 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.864019 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.864093 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.864133 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":50}
I20260806 06:45:18.864202 32625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.861322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:38829 (local address 127.14.51.66:38311)
0806 06:45:18.861393 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.861395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.861423 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:45:18.861541 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.861542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.861554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.861648 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:18.861813 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.862286 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.863384 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.863763 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.863927 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.863937 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.864099 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.864136 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:45:18.864326 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.861455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:52009 (local address 127.14.51.65:36859)
0806 06:45:18.861523 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.861525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.861643 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:45:18.861710 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.861711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.861719 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.861795 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:18.861962 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.862415 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.863588 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.863970 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.864066 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.864074 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.864226 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.864264 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:45:18.864336 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.861387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:36859 (local address 127.14.51.67:52009)
0806 06:45:18.861571 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.861579 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:18.861654 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:18.861796 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:18.861797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:18.861799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:18.861872 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.861873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.862499 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.862500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.863488 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:18.863490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.864065 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.864066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:18.864143 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.864146 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:18.864219 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:45:18.864255 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":121,"threads_started":1}
I20260806 06:45:18.864540 32422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d527d126f0bd40bc93932e14cdd1ff80" candidate_uuid: "846c9f98db9247e68238398aee53cfc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" is_pre_election: true
I20260806 06:45:18.864732 32422 raft_consensus.cc:2454] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 846c9f98db9247e68238398aee53cfc5 in term 0.
I20260806 06:45:18.865036 32496 leader_election.cc:302] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e34c0291f94a3fbb0520b8fd91b0b8, 846c9f98db9247e68238398aee53cfc5; no voters:
I20260806 06:45:18.865193 32636 raft_consensus.cc:2781] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:18.865131 32276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d527d126f0bd40bc93932e14cdd1ff80" candidate_uuid: "846c9f98db9247e68238398aee53cfc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" is_pre_election: true
I20260806 06:45:18.865262 32636 raft_consensus.cc:482] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:18.865293 32636 raft_consensus.cc:3037] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:18.865309 32276 raft_consensus.cc:2454] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 846c9f98db9247e68238398aee53cfc5 in term 0.
I20260806 06:45:18.866036 32636 raft_consensus.cc:504] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.866182 32636 leader_election.cc:288] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [CANDIDATE]: Term 1 election: Requested vote from peers 66e34c0291f94a3fbb0520b8fd91b0b8 (127.14.51.66:38311), 89aeb17087ff432792cf9cb7c3c08ee4 (127.14.51.65:36859)
I20260806 06:45:18.866393 32276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d527d126f0bd40bc93932e14cdd1ff80" candidate_uuid: "846c9f98db9247e68238398aee53cfc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89aeb17087ff432792cf9cb7c3c08ee4"
I20260806 06:45:18.866456 32276 raft_consensus.cc:3037] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:18.867209 32276 raft_consensus.cc:2454] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 846c9f98db9247e68238398aee53cfc5 in term 1.
I20260806 06:45:18.867355 32422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d527d126f0bd40bc93932e14cdd1ff80" candidate_uuid: "846c9f98db9247e68238398aee53cfc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8"
I20260806 06:45:18.867411 32496 leader_election.cc:302] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 846c9f98db9247e68238398aee53cfc5, 89aeb17087ff432792cf9cb7c3c08ee4; no voters:
I20260806 06:45:18.867440 32422 raft_consensus.cc:3037] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:18.867509 32636 raft_consensus.cc:2781] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:18.867686 32636 raft_consensus.cc:686] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 1 LEADER]: Becoming Leader. State: Replica: 846c9f98db9247e68238398aee53cfc5, State: Running, Role: LEADER
I20260806 06:45:18.867779 32636 consensus_queue.cc:237] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } }
I20260806 06:45:18.868360 32422 raft_consensus.cc:2454] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 846c9f98db9247e68238398aee53cfc5 in term 1.
I20260806 06:45:18.868692 32118 catalog_manager.cc:5495] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 846c9f98db9247e68238398aee53cfc5 (127.14.51.67). New cstate: current_term: 1 leader_uuid: "846c9f98db9247e68238398aee53cfc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "846c9f98db9247e68238398aee53cfc5" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 33283 } health_report { overall_health: HEALTHY } } }
I20260806 06:45:18.883239 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.880877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.14.51.67:33283)
0806 06:45:18.880934 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:18.880936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:18.880943 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:18.880970 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:18.880971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:18.880991 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:18.881078 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:18.881211 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.881675 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.882412 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:18.882603 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:18.882682 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:18.882861 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:18.882967 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:18.883083 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:18.883106 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 06:45:18.883125 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:45:18.885064 32422 raft_consensus.cc:1261] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 1 FOLLOWER]: Refusing update from remote peer 846c9f98db9247e68238398aee53cfc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:18.885075 32276 raft_consensus.cc:1261] T d527d126f0bd40bc93932e14cdd1ff80 P 89aeb17087ff432792cf9cb7c3c08ee4 [term 1 FOLLOWER]: Refusing update from remote peer 846c9f98db9247e68238398aee53cfc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:18.885375 32640 consensus_queue.cc:1035] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e34c0291f94a3fbb0520b8fd91b0b8" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:18.885497 32636 consensus_queue.cc:1035] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89aeb17087ff432792cf9cb7c3c08ee4" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 36859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:18.888078 32644 mvcc.cc:205] Tried to move back new op lower bound from 7315450752550379520 to 7315450752483262464. Current Snapshot: MvccSnapshot[applied={T|T < 7315450752550379520}]
I20260806 06:45:18.890162 32118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:18.890273 32118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:18.891160 32118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72a5ad90deb449dda74c2f2101445610: Sending DeleteTablet for 3 replicas of tablet d527d126f0bd40bc93932e14cdd1ff80
I20260806 06:45:18.891613 32401 tablet_service.cc:1514] Processing DeleteTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:18 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:57196
I20260806 06:45:18.891669 32546 tablet_service.cc:1514] Processing DeleteTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:18 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:57782
I20260806 06:45:18.891886 32654 tablet_replica.cc:330] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: stopping tablet replica
I20260806 06:45:18.891891 32254 tablet_service.cc:1514] Processing DeleteTablet for tablet d527d126f0bd40bc93932e14cdd1ff80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:18 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:37280
I20260806 06:45:18.891950 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32186
I20260806 06:45:18.892046 32654 raft_consensus.cc:2229] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:18.892145 32654 raft_consensus.cc:2258] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:18.892783 32655 tablet_replica.cc:330] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: stopping tablet replica
I20260806 06:45:18.892931 32655 raft_consensus.cc:2229] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:18.893129 32655 raft_consensus.cc:2258] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:18.893651 32655 ts_tablet_manager.cc:1905] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:18.893841 32654 ts_tablet_manager.cc:1905] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:18.895679 32654 ts_tablet_manager.cc:1918] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:18.895773 32654 log.cc:1196] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/d527d126f0bd40bc93932e14cdd1ff80
I20260806 06:45:18.896061 32654 ts_tablet_manager.cc:1939] T d527d126f0bd40bc93932e14cdd1ff80 P 66e34c0291f94a3fbb0520b8fd91b0b8: Deleting consensus metadata
I20260806 06:45:18.896068 32655 ts_tablet_manager.cc:1918] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:18.896167 32655 log.cc:1196] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/d527d126f0bd40bc93932e14cdd1ff80
I20260806 06:45:18.896524 32655 ts_tablet_manager.cc:1939] T d527d126f0bd40bc93932e14cdd1ff80 P 846c9f98db9247e68238398aee53cfc5: Deleting consensus metadata
W20260806 06:45:18.903793 32103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:36859 (error 108)
I20260806 06:45:18.903993 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32332
I20260806 06:45:18.904134 32103 catalog_manager.cc:4841] TS 846c9f98db9247e68238398aee53cfc5 (127.14.51.67:33283): tablet d527d126f0bd40bc93932e14cdd1ff80 (table test-table [id=9d85eaf568724c16972f7a473824f06e]) successfully deleted
I20260806 06:45:18.904220 32103 catalog_manager.cc:4841] TS 66e34c0291f94a3fbb0520b8fd91b0b8 (127.14.51.66:38311): tablet d527d126f0bd40bc93932e14cdd1ff80 (table test-table [id=9d85eaf568724c16972f7a473824f06e]) successfully deleted
I20260806 06:45:18.907825 32623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:18.904378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:36859 (local address 127.0.0.1:37292)
0806 06:45:18.907739 (+ 3361us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:18.907786 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3340}
W20260806 06:45:18.908142 32103 catalog_manager.cc:4584] TS 89aeb17087ff432792cf9cb7c3c08ee4 (127.14.51.65:36859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d527d126f0bd40bc93932e14cdd1ff80: Network error: Client connection negotiation failed: client connection to 127.14.51.65:36859: connect: Connection refused (error 111)
I20260806 06:45:18.911644 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32478
I20260806 06:45:18.916455 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32085
2026-08-06T06:45:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2047 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T06:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:18Z Disabled control of system clock
I20260806 06:45:18.943454 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34575
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:19.021407 32660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:19.021626 32660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:19.021646 32660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:19.022701 32660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:19.022733 32660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:19.022746 32660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:19.022759 32660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:19.022771 32660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:19.023983 32660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:19.024155 32660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:19.024358 32660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:19.029004 32666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.029145 32667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.029039 32669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.029246 32660 server_base.cc:1034] running on GCE node
I20260806 06:45:19.029547 32660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:19.029808 32660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:19.030977 32660 hybrid_clock.cc:648] HybridClock initialized: now 1785998719030954 us; error 41 us; skew 500 ppm
I20260806 06:45:19.032214 32660 webserver.cc:460] Webserver started at http://127.14.51.126:35447/ using document root <none> and password file <none>
I20260806 06:45:19.032445 32660 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:19.032527 32660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:19.032682 32660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:19.033681 32660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "caf0cbc28c0345a3a9c7d13f2db14490"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "8cc3d03577ff6086329fa36c5a09dc05"
server_key_iv: "0fdf6155122e3faf5ded99312e72cef5"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.034035 32660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "caf0cbc28c0345a3a9c7d13f2db14490"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "8cc3d03577ff6086329fa36c5a09dc05"
server_key_iv: "0fdf6155122e3faf5ded99312e72cef5"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.035260 32660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:19.036019 32675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.036193 32660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.036286 32660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "caf0cbc28c0345a3a9c7d13f2db14490"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "8cc3d03577ff6086329fa36c5a09dc05"
server_key_iv: "0fdf6155122e3faf5ded99312e72cef5"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.036367 32660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:19.044867 32660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:19.045110 32660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:19.045261 32660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:19.049098 32660 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:34575
I20260806 06:45:19.049130 32732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:34575 every 8 connection(s)
I20260806 06:45:19.049420 32660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:19.050283 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:19.052453 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: Bootstrap starting.
I20260806 06:45:19.053011 32733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:19.053320 32733 log.cc:824] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:19.053956 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: No bootstrap required, opened a new log
I20260806 06:45:19.055544 32733 raft_consensus.cc:348] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } }
I20260806 06:45:19.055668 32733 raft_consensus.cc:374] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:19.055689 32733 raft_consensus.cc:729] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0cbc28c0345a3a9c7d13f2db14490, State: Initialized, Role: FOLLOWER
I20260806 06:45:19.055871 32733 consensus_queue.cc:260] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } }
I20260806 06:45:19.055951 32733 raft_consensus.cc:388] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:19.056002 32733 raft_consensus.cc:482] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:19.056052 32733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:19.056564 32733 raft_consensus.cc:504] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } }
I20260806 06:45:19.056674 32733 leader_election.cc:302] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caf0cbc28c0345a3a9c7d13f2db14490; no voters:
I20260806 06:45:19.056851 32733 leader_election.cc:288] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:19.056877 32736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:19.057118 32733 sys_catalog.cc:564] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:19.057178 32736 raft_consensus.cc:686] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [term 1 LEADER]: Becoming Leader. State: Replica: caf0cbc28c0345a3a9c7d13f2db14490, State: Running, Role: LEADER
I20260806 06:45:19.057291 32736 consensus_queue.cc:237] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } }
I20260806 06:45:19.057449 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32660
I20260806 06:45:19.057577 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:19.057837 14541 external_mini_cluster.cc:1428] Setting key a6e9fa1f5dd54aac18b589467023f62f
I20260806 06:45:19.058233 32737 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } } }
I20260806 06:45:19.058318 32737 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:19.058475 32738 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caf0cbc28c0345a3a9c7d13f2db14490. Latest consensus state: current_term: 1 leader_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf0cbc28c0345a3a9c7d13f2db14490" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34575 } } }
I20260806 06:45:19.058542 32738 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:19.058645 32750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:19.059121 32750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:19.060856 32750 catalog_manager.cc:1347] Generated new cluster ID: 47533c8ba8254ef58e6d70670def7286
I20260806 06:45:19.060912 32750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:19.063735 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.058474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.14.51.126:34575)
0806 06:45:19.058642 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.058651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.058989 (+ 338us) server_negotiation.cc:407] Connection header received
0806 06:45:19.059168 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.059175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.059336 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.059682 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:19.059746 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.061003 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.061807 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.062217 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.062349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.063028 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:19.063031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:19.063032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:19.063265 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:19.063318 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:19.063330 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:19.063418 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.063490 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.063561 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":74,"threads_started":1}
I20260806 06:45:19.069101 32750 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:19.069564 32750 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:19.076494 32750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: Generated new TSK 0
I20260806 06:45:19.076694 32750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:19.077792 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--builtin_ntp_servers=127.14.51.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:19.177285 32757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:19.177506 32757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:19.177543 32757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:19.178637 32757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:19.178728 32757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:19.178772 32757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:19.180169 32757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:19.180420 32757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:19.180696 32757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:19.186512 32763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.186520 32762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.186527 32765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.186753 32757 server_base.cc:1034] running on GCE node
I20260806 06:45:19.186960 32757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:19.187245 32757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:19.188412 32757 hybrid_clock.cc:648] HybridClock initialized: now 1785998719188389 us; error 40 us; skew 500 ppm
I20260806 06:45:19.189651 32757 webserver.cc:460] Webserver started at http://127.14.51.65:38681/ using document root <none> and password file <none>
I20260806 06:45:19.189889 32757 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:19.189966 32757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:19.190102 32757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:19.191200 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "7f41b10cde9e40fd868fdd6e38e8639b"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "2850101c448e2cd907d56e415f2f84ea"
server_key_iv: "40845a1f0f346ea83464ab54c71604a0"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.191566 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "7f41b10cde9e40fd868fdd6e38e8639b"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "2850101c448e2cd907d56e415f2f84ea"
server_key_iv: "40845a1f0f346ea83464ab54c71604a0"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.192963 32757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:19.193775 305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.193972 32757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:19.194096 32757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "7f41b10cde9e40fd868fdd6e38e8639b"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "2850101c448e2cd907d56e415f2f84ea"
server_key_iv: "40845a1f0f346ea83464ab54c71604a0"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.194185 32757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:19.201910 32757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:19.202168 32757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:19.202314 32757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:19.202534 32757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:19.202828 32757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:19.202885 32757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.202919 32757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:19.202944 32757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.209107 32757 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:43497
I20260806 06:45:19.209148 425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:43497 every 8 connection(s)
I20260806 06:45:19.209473 32757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:19.213323 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 32757
I20260806 06:45:19.213474 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:19.213680 14541 external_mini_cluster.cc:1428] Setting key 027a3a366ea406f32dff446b7505aec0
I20260806 06:45:19.214692 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--builtin_ntp_servers=127.14.51.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:45:19.215559 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.210205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43257 (local address 127.14.51.126:34575)
0806 06:45:19.210293 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.210303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.210575 (+ 272us) server_negotiation.cc:407] Connection header received
0806 06:45:19.211205 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.211206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.211219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.211284 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:19.212073 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.212547 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.213865 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.214179 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.214245 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.215280 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:19.215281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:19.215281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:19.215287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:19.215301 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:19.215305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:19.215342 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.215461 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.215487 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 06:45:19.215963 428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.210191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34575 (local address 127.14.51.65:43257)
0806 06:45:19.210436 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.210476 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.211061 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.211391 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.211394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.211404 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:19.211989 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.211994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.212649 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.212650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.213768 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.213771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.214357 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.214359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.214515 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.215098 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:19.215099 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:19.215151 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:19.215180 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:19.215383 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:19.215387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:19.215390 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.215667 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.215773 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
I20260806 06:45:19.216387 426 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34575
I20260806 06:45:19.216472 426 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:19.216637 426 heartbeater.cc:511] Master 127.14.51.126:34575 requested a full tablet report, sending...
I20260806 06:45:19.217115 32695 ts_manager.cc:194] Registered new tserver with Master: 7f41b10cde9e40fd868fdd6e38e8639b (127.14.51.65:43497)
I20260806 06:45:19.217722 32695 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.65:43257
W20260806 06:45:19.293910 429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:19.294085 429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:19.294103 429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:19.295264 429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:19.295302 429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:19.295325 429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:19.296705 429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:19.296941 429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:19.297204 429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:19.301860 435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.301868 438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.302008 429 server_base.cc:1034] running on GCE node
W20260806 06:45:19.301964 436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.302225 429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:19.302484 429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:19.303673 429 hybrid_clock.cc:648] HybridClock initialized: now 1785998719303663 us; error 30 us; skew 500 ppm
I20260806 06:45:19.304912 429 webserver.cc:460] Webserver started at http://127.14.51.66:43743/ using document root <none> and password file <none>
I20260806 06:45:19.305168 429 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:19.305241 429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:19.305395 429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:19.306435 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "f8edcf4c9a774cbb9e891fbce408045e"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "6b384967476d31c1826b6953d62b56d8"
server_key_iv: "f462f04efe0bf8d188fca5b04cf95b13"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.306792 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "f8edcf4c9a774cbb9e891fbce408045e"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "6b384967476d31c1826b6953d62b56d8"
server_key_iv: "f462f04efe0bf8d188fca5b04cf95b13"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.308125 429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:19.308844 445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.309022 429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:19.309121 429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "f8edcf4c9a774cbb9e891fbce408045e"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "6b384967476d31c1826b6953d62b56d8"
server_key_iv: "f462f04efe0bf8d188fca5b04cf95b13"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.309211 429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:19.368640 429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:19.368981 429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:19.369143 429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:19.369393 429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:19.369738 429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:19.369808 429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.369904 429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:19.369937 429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.375589 429 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:33267
I20260806 06:45:19.375630 565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:33267 every 8 connection(s)
I20260806 06:45:19.375952 429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:19.380833 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.376639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:36097 (local address 127.14.51.126:34575)
0806 06:45:19.376724 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.376726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.376921 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:45:19.377374 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.377375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.377385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.377437 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:19.378170 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.378689 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.379618 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.379794 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.379849 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.380599 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:19.380600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:19.380601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:19.380604 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:19.380618 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:19.380622 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:19.380664 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.380754 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.380777 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:45:19.381003 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.376588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34575 (local address 127.14.51.66:36097)
0806 06:45:19.376798 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.376831 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.377280 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.377527 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.377530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.377538 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:19.378078 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.378082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.378772 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.378773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.379544 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.379545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.379870 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.379871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.379968 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.380454 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:19.380455 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:19.380492 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:19.380513 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:19.380677 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:19.380681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:19.380683 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.380734 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.380807 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
I20260806 06:45:19.381266 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 429
I20260806 06:45:19.381395 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:19.381531 566 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34575
I20260806 06:45:19.381625 566 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:19.381624 14541 external_mini_cluster.cc:1428] Setting key 4112634d6d471beba8414379fc017cf2
I20260806 06:45:19.381804 566 heartbeater.cc:511] Master 127.14.51.126:34575 requested a full tablet report, sending...
I20260806 06:45:19.382195 32695 ts_manager.cc:194] Registered new tserver with Master: f8edcf4c9a774cbb9e891fbce408045e (127.14.51.66:33267)
I20260806 06:45:19.382551 32695 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.66:36097
I20260806 06:45:19.382579 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--builtin_ntp_servers=127.14.51.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:19.462764 569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:19.463001 569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:19.463021 569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:19.464202 569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:19.464243 569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:19.464269 569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:19.465598 569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:34575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:19.465822 569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:19.466094 569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:19.471068 579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.471134 569 server_base.cc:1034] running on GCE node
W20260806 06:45:19.471078 574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:19.471081 577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:19.471448 569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:19.471624 569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:19.472791 569 hybrid_clock.cc:648] HybridClock initialized: now 1785998719472766 us; error 63 us; skew 500 ppm
I20260806 06:45:19.473963 569 webserver.cc:460] Webserver started at http://127.14.51.67:38581/ using document root <none> and password file <none>
I20260806 06:45:19.474190 569 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:19.474260 569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:19.474391 569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:19.475462 569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "66d2812dd4c94eb3a060b242a78045fe"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "01f6941e1dedba6d5344d8be34d7fc7f"
server_key_iv: "34acac5bd668c7920f6c1e38d24a82b9"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.475828 569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "66d2812dd4c94eb3a060b242a78045fe"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "01f6941e1dedba6d5344d8be34d7fc7f"
server_key_iv: "34acac5bd668c7920f6c1e38d24a82b9"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.477087 569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:19.477766 586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.477929 569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:19.478027 569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "66d2812dd4c94eb3a060b242a78045fe"
format_stamp: "Formatted at 2026-08-06 06:45:19 on dist-test-slave-1xrh"
server_key: "01f6941e1dedba6d5344d8be34d7fc7f"
server_key_iv: "34acac5bd668c7920f6c1e38d24a82b9"
server_key_version: "encryptionkey@0"
I20260806 06:45:19.478106 569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:19.484251 569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:19.484503 569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:19.484647 569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:19.484850 569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:19.485143 569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:19.485200 569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.485237 569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:19.485265 569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:19.491161 569 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:38141
I20260806 06:45:19.491207 706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:38141 every 8 connection(s)
I20260806 06:45:19.491501 569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:19.496052 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.492155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:36425 (local address 127.14.51.126:34575)
0806 06:45:19.492216 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.492218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.492465 (+ 247us) server_negotiation.cc:407] Connection header received
0806 06:45:19.492857 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.492858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.492869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.492931 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:19.493511 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.493977 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.494843 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.495016 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.495063 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.495824 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:19.495826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:19.495826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:19.495829 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:19.495842 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:19.495846 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:19.495888 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.495968 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.495993 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:45:19.496233 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.492091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34575 (local address 127.14.51.67:36425)
0806 06:45:19.492314 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.492353 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.492778 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.492976 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.492979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.492985 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:19.493426 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.493430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.494056 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.494057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.494770 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.494771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.495089 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.495090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.495262 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.495693 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:19.495693 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:19.495717 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:19.495735 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:19.495900 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:19.495904 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:19.495906 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.495954 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.496023 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260806 06:45:19.496685 707 heartbeater.cc:347] Connected to a master server at 127.14.51.126:34575
I20260806 06:45:19.496778 707 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:19.497033 707 heartbeater.cc:511] Master 127.14.51.126:34575 requested a full tablet report, sending...
I20260806 06:45:19.497113 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 569
I20260806 06:45:19.497273 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:19.497553 14541 external_mini_cluster.cc:1428] Setting key 2bdcbe3437c79047796ef2941efdd655
I20260806 06:45:19.497541 32695 ts_manager.cc:194] Registered new tserver with Master: 66d2812dd4c94eb3a060b242a78045fe (127.14.51.67:38141)
I20260806 06:45:19.497926 32695 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.67:36425
I20260806 06:45:19.498435 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:19.502331 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.499854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.14.51.126:34575)
0806 06:45:19.499929 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.499931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.499998 (+ 67us) server_negotiation.cc:407] Connection header received
0806 06:45:19.500042 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.500042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.500051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.500096 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:19.500241 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.500668 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.501348 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.501517 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.501563 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.502100 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:19.502101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:19.502101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:19.502105 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:19.502117 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:19.502120 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:19.502163 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.502210 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.502230 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:45:19.503882 32695 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:19.505379 32695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:19.511575 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:38141 (local address 127.0.0.1:33458)
0806 06:45:19.507836 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.507898 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.507969 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.508173 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.508176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.508179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:19.508255 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.508257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509579 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.509580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.510505 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.510508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511286 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.511287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.511373 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.511387 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.511449 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.511501 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":25,"thread_start_us":82,"threads_started":1}
I20260806 06:45:19.511677 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.14.51.65:43497)
0806 06:45:19.508063 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.508067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.508101 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:45:19.508202 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.508204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.508218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.508446 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:19.508506 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509204 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.510605 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511292 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.511386 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.511473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.511548 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.511584 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":98,"threads_started":1}
I20260806 06:45:19.511849 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43497 (local address 127.0.0.1:52822)
0806 06:45:19.507737 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.508063 (+ 326us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.508097 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.508318 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.508320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.508322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:19.508368 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.508370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509340 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.509341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.510508 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.510510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511403 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.511404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.511462 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.511467 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.511534 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.511573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":45,"threads_started":1}
I20260806 06:45:19.512092 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.14.51.67:38141)
0806 06:45:19.507969 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.507974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.507985 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:45:19.508052 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.508054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.508069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.508147 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:19.508343 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509488 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.510589 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511200 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.511268 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.511341 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.511963 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.512017 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260806 06:45:19.512202 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33267 (local address 127.0.0.1:37152)
0806 06:45:19.507957 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.507966 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.508007 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.508431 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.508431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.508434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:19.508467 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.508468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509768 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.509769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.510797 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.510799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511698 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.511700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.511846 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.511850 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.511892 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.511938 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":71,"threads_started":1}
I20260806 06:45:19.512593 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.507809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.14.51.66:33267)
0806 06:45:19.508251 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.508256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.508266 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:19.508302 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.508305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.508322 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.508549 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:19.508605 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.509674 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.510890 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.511601 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.512356 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.512445 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.512470 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.512510 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":117,"threads_started":1}
I20260806 06:45:19.512575 358 tablet_service.cc:1467] Processing CreateTablet for tablet b887591dfc274eebbdaa144b6b293b5c (DEFAULT_TABLE table=test-table [id=218c679eb3c44e7b8779a68193cc678f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:19.512691 638 tablet_service.cc:1467] Processing CreateTablet for tablet b887591dfc274eebbdaa144b6b293b5c (DEFAULT_TABLE table=test-table [id=218c679eb3c44e7b8779a68193cc678f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:19.512945 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b887591dfc274eebbdaa144b6b293b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:19.513072 638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b887591dfc274eebbdaa144b6b293b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:19.513172 497 tablet_service.cc:1467] Processing CreateTablet for tablet b887591dfc274eebbdaa144b6b293b5c (DEFAULT_TABLE table=test-table [id=218c679eb3c44e7b8779a68193cc678f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:19.513463 497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b887591dfc274eebbdaa144b6b293b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:19.515906 723 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Bootstrap starting.
I20260806 06:45:19.516280 724 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Bootstrap starting.
I20260806 06:45:19.516673 723 tablet_bootstrap.cc:654] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:19.516788 725 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Bootstrap starting.
I20260806 06:45:19.516880 724 tablet_bootstrap.cc:654] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:19.517179 723 log.cc:824] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:19.517198 724 log.cc:824] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:19.517369 725 tablet_bootstrap.cc:654] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:19.517693 725 log.cc:824] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:19.517967 724 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: No bootstrap required, opened a new log
I20260806 06:45:19.518059 724 ts_tablet_manager.cc:1397] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:19.518129 723 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: No bootstrap required, opened a new log
I20260806 06:45:19.518198 723 ts_tablet_manager.cc:1397] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:19.518446 725 tablet_bootstrap.cc:492] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: No bootstrap required, opened a new log
I20260806 06:45:19.518509 725 ts_tablet_manager.cc:1397] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:19.520089 724 raft_consensus.cc:348] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.520242 724 raft_consensus.cc:374] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:19.520287 724 raft_consensus.cc:729] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f41b10cde9e40fd868fdd6e38e8639b, State: Initialized, Role: FOLLOWER
I20260806 06:45:19.520393 724 consensus_queue.cc:260] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.520646 724 ts_tablet_manager.cc:1428] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:19.520789 426 heartbeater.cc:503] Master 127.14.51.126:34575 was elected leader, sending a full tablet report...
I20260806 06:45:19.520793 723 raft_consensus.cc:348] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.520948 723 raft_consensus.cc:374] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:19.520980 723 raft_consensus.cc:729] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d2812dd4c94eb3a060b242a78045fe, State: Initialized, Role: FOLLOWER
I20260806 06:45:19.521103 723 consensus_queue.cc:260] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.521337 723 ts_tablet_manager.cc:1428] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:19.521294 725 raft_consensus.cc:348] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.521440 725 raft_consensus.cc:374] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:19.521464 707 heartbeater.cc:503] Master 127.14.51.126:34575 was elected leader, sending a full tablet report...
I20260806 06:45:19.521479 725 raft_consensus.cc:729] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8edcf4c9a774cbb9e891fbce408045e, State: Initialized, Role: FOLLOWER
I20260806 06:45:19.521591 725 consensus_queue.cc:260] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.521883 725 ts_tablet_manager.cc:1428] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:19.521909 566 heartbeater.cc:503] Master 127.14.51.126:34575 was elected leader, sending a full tablet report...
W20260806 06:45:19.626437 567 tablet.cc:2369] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:45:19.710256 427 tablet.cc:2369] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:45:19.742012 708 tablet.cc:2369] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:19.784202 729 raft_consensus.cc:482] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:19.784318 729 raft_consensus.cc:504] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.784770 729 leader_election.cc:288] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66d2812dd4c94eb3a060b242a78045fe (127.14.51.67:38141), f8edcf4c9a774cbb9e891fbce408045e (127.14.51.66:33267)
I20260806 06:45:19.787819 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.785106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:43103 (local address 127.14.51.66:33267)
0806 06:45:19.785156 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.785158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.785264 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:45:19.785349 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.785350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.785367 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.785436 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:19.785619 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.786078 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787078 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.787447 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787550 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.787560 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.787713 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.787748 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:45:19.787835 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.785091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:48397 (local address 127.14.51.67:38141)
0806 06:45:19.785138 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.785140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.785175 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:45:19.785295 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.785296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.785307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.785393 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:19.785553 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.786036 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787095 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.787476 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787565 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.787573 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.787731 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.787770 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:45:19.787846 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.785041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:33267 (local address 127.14.51.65:43103)
0806 06:45:19.785172 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.785182 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.785277 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.785461 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.785462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.785463 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:19.785536 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.785538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.786174 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.786175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.786977 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.786979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.787559 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.787635 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.787639 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.787708 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.787748 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":17,"thread_start_us":70,"threads_started":1}
I20260806 06:45:19.787850 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.784962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:38141 (local address 127.14.51.65:48397)
0806 06:45:19.785110 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.785123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:19.785205 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:19.785397 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:19.785398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:19.785400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:19.785471 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.785472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.786128 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.786129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.786995 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:19.786997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.787566 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.787566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:19.787644 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.787648 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:19.787713 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:45:19.787751 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260806 06:45:19.788196 518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b887591dfc274eebbdaa144b6b293b5c" candidate_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8edcf4c9a774cbb9e891fbce408045e" is_pre_election: true
I20260806 06:45:19.788367 518 raft_consensus.cc:2454] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f41b10cde9e40fd868fdd6e38e8639b in term 0.
I20260806 06:45:19.788412 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b887591dfc274eebbdaa144b6b293b5c" candidate_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d2812dd4c94eb3a060b242a78045fe" is_pre_election: true
I20260806 06:45:19.788591 658 raft_consensus.cc:2454] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f41b10cde9e40fd868fdd6e38e8639b in term 0.
I20260806 06:45:19.788630 306 leader_election.cc:302] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f41b10cde9e40fd868fdd6e38e8639b, f8edcf4c9a774cbb9e891fbce408045e; no voters:
I20260806 06:45:19.788774 729 raft_consensus.cc:2781] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:19.788851 729 raft_consensus.cc:482] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:19.788884 729 raft_consensus.cc:3037] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:19.789599 729 raft_consensus.cc:504] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.789780 729 leader_election.cc:288] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [CANDIDATE]: Term 1 election: Requested vote from peers 66d2812dd4c94eb3a060b242a78045fe (127.14.51.67:38141), f8edcf4c9a774cbb9e891fbce408045e (127.14.51.66:33267)
I20260806 06:45:19.789938 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b887591dfc274eebbdaa144b6b293b5c" candidate_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d2812dd4c94eb3a060b242a78045fe"
I20260806 06:45:19.789951 518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b887591dfc274eebbdaa144b6b293b5c" candidate_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8edcf4c9a774cbb9e891fbce408045e"
I20260806 06:45:19.790027 518 raft_consensus.cc:3037] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:19.790051 658 raft_consensus.cc:3037] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:19.790798 518 raft_consensus.cc:2454] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f41b10cde9e40fd868fdd6e38e8639b in term 1.
I20260806 06:45:19.790978 306 leader_election.cc:302] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f41b10cde9e40fd868fdd6e38e8639b, f8edcf4c9a774cbb9e891fbce408045e; no voters:
I20260806 06:45:19.790984 658 raft_consensus.cc:2454] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f41b10cde9e40fd868fdd6e38e8639b in term 1.
I20260806 06:45:19.791059 729 raft_consensus.cc:2781] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:19.791235 729 raft_consensus.cc:686] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 1 LEADER]: Becoming Leader. State: Replica: 7f41b10cde9e40fd868fdd6e38e8639b, State: Running, Role: LEADER
I20260806 06:45:19.791330 729 consensus_queue.cc:237] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } }
I20260806 06:45:19.792166 32695 catalog_manager.cc:5495] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f41b10cde9e40fd868fdd6e38e8639b (127.14.51.65). New cstate: current_term: 1 leader_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f41b10cde9e40fd868fdd6e38e8639b" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 43497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 } health_report { overall_health: UNKNOWN } } }
I20260806 06:45:19.866750 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.864570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.14.51.65:43497)
0806 06:45:19.864631 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:19.864633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:19.864642 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:19.864690 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:19.864691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:19.864712 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:19.864776 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:19.864909 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.865363 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.866103 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:19.866283 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:19.866332 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:19.866472 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:19.866572 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:19.866631 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:19.866675 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:45:19.866692 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:45:19.868569 658 raft_consensus.cc:1261] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 1 FOLLOWER]: Refusing update from remote peer 7f41b10cde9e40fd868fdd6e38e8639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:19.868614 518 raft_consensus.cc:1261] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 1 FOLLOWER]: Refusing update from remote peer 7f41b10cde9e40fd868fdd6e38e8639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:19.868970 734 consensus_queue.cc:1035] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8edcf4c9a774cbb9e891fbce408045e" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 33267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:19.869115 729 consensus_queue.cc:1035] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d2812dd4c94eb3a060b242a78045fe" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:19.872107 737 mvcc.cc:205] Tried to move back new op lower bound from 7315450756579053568 to 7315450756265811968. Current Snapshot: MvccSnapshot[applied={T|T < 7315450756579053568 or (T in {7315450756579053568})}]
I20260806 06:45:19.872222 738 mvcc.cc:205] Tried to move back new op lower bound from 7315450756579053568 to 7315450756265811968. Current Snapshot: MvccSnapshot[applied={T|T < 7315450756579053568}]
I20260806 06:45:19.873667 32694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:19.873775 32694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:19.874632 32694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caf0cbc28c0345a3a9c7d13f2db14490: Sending DeleteTablet for 3 replicas of tablet b887591dfc274eebbdaa144b6b293b5c
I20260806 06:45:19.874938 638 tablet_service.cc:1514] Processing DeleteTablet for tablet b887591dfc274eebbdaa144b6b293b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:19 UTC) from {username='slave'} at 127.0.0.1:33458
I20260806 06:45:19.874950 358 tablet_service.cc:1514] Processing DeleteTablet for tablet b887591dfc274eebbdaa144b6b293b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:19 UTC) from {username='slave'} at 127.0.0.1:52822
I20260806 06:45:19.874974 497 tablet_service.cc:1514] Processing DeleteTablet for tablet b887591dfc274eebbdaa144b6b293b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:19 UTC) from {username='slave'} at 127.0.0.1:37152
I20260806 06:45:19.875198 748 tablet_replica.cc:330] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: stopping tablet replica
I20260806 06:45:19.875298 748 raft_consensus.cc:2229] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:19.875401 748 raft_consensus.cc:2258] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:19.875430 750 tablet_replica.cc:330] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: stopping tablet replica
I20260806 06:45:19.875609 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32757
I20260806 06:45:19.875747 750 raft_consensus.cc:2229] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:19.875842 748 ts_tablet_manager.cc:1905] T b887591dfc274eebbdaa144b6b293b5c P 7f41b10cde9e40fd868fdd6e38e8639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:19.878412 750 raft_consensus.cc:2258] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:19.878590 749 tablet_replica.cc:330] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: stopping tablet replica
I20260806 06:45:19.878669 749 raft_consensus.cc:2229] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:19.878780 749 raft_consensus.cc:2258] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:19.879196 750 ts_tablet_manager.cc:1905] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:19.879207 749 ts_tablet_manager.cc:1905] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:19.881186 749 ts_tablet_manager.cc:1918] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:19.881186 750 ts_tablet_manager.cc:1918] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:19.881261 749 log.cc:1196] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/b887591dfc274eebbdaa144b6b293b5c
I20260806 06:45:19.881261 750 log.cc:1196] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/b887591dfc274eebbdaa144b6b293b5c
I20260806 06:45:19.881533 749 ts_tablet_manager.cc:1939] T b887591dfc274eebbdaa144b6b293b5c P 66d2812dd4c94eb3a060b242a78045fe: Deleting consensus metadata
I20260806 06:45:19.881541 750 ts_tablet_manager.cc:1939] T b887591dfc274eebbdaa144b6b293b5c P f8edcf4c9a774cbb9e891fbce408045e: Deleting consensus metadata
I20260806 06:45:19.881927 32677 catalog_manager.cc:4841] TS f8edcf4c9a774cbb9e891fbce408045e (127.14.51.66:33267): tablet b887591dfc274eebbdaa144b6b293b5c (table test-table [id=218c679eb3c44e7b8779a68193cc678f]) successfully deleted
I20260806 06:45:19.881953 32678 catalog_manager.cc:4841] TS 66d2812dd4c94eb3a060b242a78045fe (127.14.51.67:38141): tablet b887591dfc274eebbdaa144b6b293b5c (table test-table [id=218c679eb3c44e7b8779a68193cc678f]) successfully deleted
W20260806 06:45:19.882843 32679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:43497 (error 108)
I20260806 06:45:19.883390 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:19.883254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:43497 (local address 127.0.0.1:52832)
0806 06:45:19.883312 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:19.883354 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:45:19.883417 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 429
W20260806 06:45:19.883533 32679 catalog_manager.cc:4584] TS 7f41b10cde9e40fd868fdd6e38e8639b (127.14.51.65:43497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b887591dfc274eebbdaa144b6b293b5c: Network error: Client connection negotiation failed: client connection to 127.14.51.65:43497: connect: Connection refused (error 111)
I20260806 06:45:19.890496 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 569
I20260806 06:45:19.895282 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 32660
2026-08-06T06:45:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (977 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:19 dist-test-slave-1xrh krb5kdc[754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:19 dist-test-slave-1xrh krb5kdc[754](info): set up 2 sockets
Aug 06 06:45:19 dist-test-slave-1xrh krb5kdc[754](info): commencing operation
krb5kdc: starting...
W20260806 06:45:20.954579 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:21.074374 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44777
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:44777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:21.153249 768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:21.153491 768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:21.153528 768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:21.154722 768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:21.154801 768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:21.154827 768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:21.154874 768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:21.154893 768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:21.156399 768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:44777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:21.156679 768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:21.156899 768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:21.161944 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.161967 774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.161983 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.162147 768 server_base.cc:1034] running on GCE node
I20260806 06:45:21.162447 768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:21.162696 768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:21.163852 768 hybrid_clock.cc:648] HybridClock initialized: now 1785998721163824 us; error 43 us; skew 500 ppm
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:21.166241 768 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:21.166607 768 webserver.cc:460] Webserver started at http://127.14.51.126:35689/ using document root <none> and password file <none>
I20260806 06:45:21.166778 768 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:21.166817 768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:21.166901 768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:21.167950 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "64d1935da5f6451e9624a7272def8e38"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "3b68e8841e718c1b4146c9768e18e5b6"
server_key_iv: "d10dde10905ca849e70e95a7d3e24d97"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.168351 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "64d1935da5f6451e9624a7272def8e38"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "3b68e8841e718c1b4146c9768e18e5b6"
server_key_iv: "d10dde10905ca849e70e95a7d3e24d97"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.169605 768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:21.170274 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.170478 768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:21.170563 768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "64d1935da5f6451e9624a7272def8e38"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "3b68e8841e718c1b4146c9768e18e5b6"
server_key_iv: "d10dde10905ca849e70e95a7d3e24d97"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.170616 768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:21.180595 768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:21.182436 768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:21.182545 768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:21.186702 768 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:44777
I20260806 06:45:21.186789 841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:44777 every 8 connection(s)
I20260806 06:45:21.187107 768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:21.188024 842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:21.188555 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 768
I20260806 06:45:21.188648 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:21.188869 14541 external_mini_cluster.cc:1428] Setting key 1142c2ae345ba6316b6ce35ca432cf9c
I20260806 06:45:21.190995 842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: Bootstrap starting.
I20260806 06:45:21.191781 842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:21.192116 842 log.cc:824] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:21.192878 842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: No bootstrap required, opened a new log
I20260806 06:45:21.195438 842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } }
I20260806 06:45:21.195596 842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:21.195645 842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d1935da5f6451e9624a7272def8e38, State: Initialized, Role: FOLLOWER
I20260806 06:45:21.195773 842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } }
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:21.195883 842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:21.195930 842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:21.196017 842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:21.196695 842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } }
I20260806 06:45:21.196799 842 leader_election.cc:302] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64d1935da5f6451e9624a7272def8e38; no voters:
I20260806 06:45:21.197014 842 leader_election.cc:288] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:21.197044 848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:21.197154 848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [term 1 LEADER]: Becoming Leader. State: Replica: 64d1935da5f6451e9624a7272def8e38, State: Running, Role: LEADER
I20260806 06:45:21.197280 842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:21.197583 848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } }
I20260806 06:45:21.198810 849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d1935da5f6451e9624a7272def8e38. Latest consensus state: current_term: 1 leader_uuid: "64d1935da5f6451e9624a7272def8e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } } }
I20260806 06:45:21.198912 849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:21.199290 851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64d1935da5f6451e9624a7272def8e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d1935da5f6451e9624a7272def8e38" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44777 } } }
W20260806 06:45:21.199419 864 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 06:45:21.199489 851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38 [sys.catalog]: This master's current role is: LEADER
W20260806 06:45:21.199556 864 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:21.199647 865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:21.199795 865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:21.200503 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.189419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.14.51.126:44777)
0806 06:45:21.189622 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.189631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.189665 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:45:21.190184 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.190192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.190250 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.190396 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:21.190975 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.192424 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.193259 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.193832 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.194103 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.196437 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:21.196440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:21.196440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:21.196449 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:21.198465 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.198905 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.198907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.198908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.199034 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.199295 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.199295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.199296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.199618 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:21.199758 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.200184 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.200298 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260806 06:45:21.201216 865 catalog_manager.cc:1347] Generated new cluster ID: bb2c071b6f364f9fa3fbf9dcc42f14e1
I20260806 06:45:21.201292 865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:21.224159 865 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:21.224679 865 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:21.230350 865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: Generated new TSK 0
I20260806 06:45:21.230523 865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:45:21.267418 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:21.350469 870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:21.350653 870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:21.350674 870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:21.351951 870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:21.351995 870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:21.352022 870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:21.353456 870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:21.353705 870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:21.353947 870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:21.358791 877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.358799 878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.358891 870 server_base.cc:1034] running on GCE node
W20260806 06:45:21.358830 880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.359234 870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:21.359453 870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:21.360605 870 hybrid_clock.cc:648] HybridClock initialized: now 1785998721360573 us; error 47 us; skew 500 ppm
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:21.363281 870 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:45:21.363718 870 webserver.cc:460] Webserver started at http://127.14.51.65:46627/ using document root <none> and password file <none>
I20260806 06:45:21.363993 870 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:21.364074 870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:21.364236 870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:21.365267 870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "4965d268972e4850b421db83d0385483"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "40696c28f4b92386169a522854f0b334"
server_key_iv: "01586328e24e652fc0e6e5df45c321e4"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.365634 870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "4965d268972e4850b421db83d0385483"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "40696c28f4b92386169a522854f0b334"
server_key_iv: "01586328e24e652fc0e6e5df45c321e4"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.366922 870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:21.367730 888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.367900 870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.368002 870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "4965d268972e4850b421db83d0385483"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "40696c28f4b92386169a522854f0b334"
server_key_iv: "01586328e24e652fc0e6e5df45c321e4"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.368088 870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:21.399683 870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:21.402089 870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:21.402262 870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:21.402513 870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:21.402827 870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:21.402894 870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.402936 870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:21.402957 870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.408694 870 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:46073
I20260806 06:45:21.408746 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:46073 every 8 connection(s)
I20260806 06:45:21.409055 870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:21.412379 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 870
I20260806 06:45:21.412518 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:21.412735 14541 external_mini_cluster.cc:1428] Setting key 6a434602de9309ac3cb078027eda991e
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:21.418792 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.409768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:33079 (local address 127.14.51.126:44777)
0806 06:45:21.409832 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.409834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.410039 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:45:21.410522 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.410522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.410532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.410591 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:21.411608 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.412110 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.413303 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.413590 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.413680 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.415998 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:21.416000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:21.416000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:21.416005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:21.417308 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.417788 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.417788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.417789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.417839 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.418137 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.418138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.418138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.418235 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:21.418283 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.418652 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.418719 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:45:21.419086 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.409683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44777 (local address 127.14.51.65:33079)
0806 06:45:21.409901 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.409943 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.410431 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.410690 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.410693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.411315 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:21.411520 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.411524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.412191 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.412191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.413215 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.413218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.413692 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.413693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.413819 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.414312 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:21.414319 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:21.415903 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:21.417386 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.417394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.417396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.417708 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.417924 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.417925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.417926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.418066 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.418303 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:21.418310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:21.418493 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.418637 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.418829 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260806 06:45:21.419629 1010 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44777
I20260806 06:45:21.419718 1010 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:21.419883 1010 heartbeater.cc:511] Master 127.14.51.126:44777 requested a full tablet report, sending...
I20260806 06:45:21.420423 804 ts_manager.cc:194] Registered new tserver with Master: 4965d268972e4850b421db83d0385483 (127.14.51.65:46073)
I20260806 06:45:21.421052 804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:33079
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:45:21.439844 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:21.519487 1015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:21.519672 1015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:21.519694 1015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:21.520817 1015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:21.520859 1015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:21.520887 1015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:21.522316 1015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:21.522545 1015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:21.522763 1015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:21.527676 1023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.527683 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.527694 1021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.527908 1015 server_base.cc:1034] running on GCE node
I20260806 06:45:21.528123 1015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:21.528352 1015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:21.529503 1015 hybrid_clock.cc:648] HybridClock initialized: now 1785998721529472 us; error 46 us; skew 500 ppm
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:21.531986 1015 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:45:21.532442 1015 webserver.cc:460] Webserver started at http://127.14.51.66:41279/ using document root <none> and password file <none>
I20260806 06:45:21.532671 1015 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:21.532716 1015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:21.532881 1015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:21.533885 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "577212ed1c0c4c31b981ef8f05e37b51"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "5b9e9c31e971031b55bfda5e557d2372"
server_key_iv: "425824c5fbc5b5a36830c563bd138bcc"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.534232 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "577212ed1c0c4c31b981ef8f05e37b51"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "5b9e9c31e971031b55bfda5e557d2372"
server_key_iv: "425824c5fbc5b5a36830c563bd138bcc"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.535547 1015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:45:21.536257 1030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.536454 1015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.536530 1015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "577212ed1c0c4c31b981ef8f05e37b51"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "5b9e9c31e971031b55bfda5e557d2372"
server_key_iv: "425824c5fbc5b5a36830c563bd138bcc"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.536580 1015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:21.546124 1015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:21.548341 1015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:21.548446 1015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:21.548614 1015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:21.548834 1015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:21.548864 1015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.548897 1015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:21.548908 1015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.554306 1015 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:42151
I20260806 06:45:21.554370 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:42151 every 8 connection(s)
I20260806 06:45:21.554677 1015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:21.563421 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.555538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:51861 (local address 127.14.51.126:44777)
0806 06:45:21.555611 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.555613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.555849 (+ 236us) server_negotiation.cc:407] Connection header received
0806 06:45:21.556273 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.556274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.556283 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.556338 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:21.557186 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.557668 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.558698 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.558870 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.558919 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.560864 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:21.560866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:21.560866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:21.560872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:21.562283 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.562624 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.562625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.562625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.562674 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.562881 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.562881 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.562881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.562975 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:21.563025 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.563300 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.563359 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:45:21.563642 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.555531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44777 (local address 127.14.51.66:51861)
0806 06:45:21.555722 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.555760 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.556191 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.556384 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.556387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.556908 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:21.557109 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.557114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.557749 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.557750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.558622 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.558624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.558942 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.558943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.559041 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.559512 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:21.559520 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:21.560777 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:21.562358 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.562363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.562364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.562557 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.562736 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.562737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.562737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.562824 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.563040 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:21.563044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:21.563203 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.563283 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.563393 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":89,"threads_started":1}
I20260806 06:45:21.564038 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1015
I20260806 06:45:21.564181 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:21.564226 1153 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44777
I20260806 06:45:21.564314 1153 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:21.564409 14541 external_mini_cluster.cc:1428] Setting key 71b4b61bc35b29317f95f0747f570958
I20260806 06:45:21.564513 1153 heartbeater.cc:511] Master 127.14.51.126:44777 requested a full tablet report, sending...
I20260806 06:45:21.564916 802 ts_manager.cc:194] Registered new tserver with Master: 577212ed1c0c4c31b981ef8f05e37b51 (127.14.51.66:42151)
I20260806 06:45:21.565415 802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:51861
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:45:21.589641 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:21.668893 1158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:21.669080 1158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:21.669102 1158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:21.670352 1158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:21.670392 1158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:21.670440 1158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:21.672011 1158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:21.672246 1158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:21.672473 1158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:21.677556 1164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.677688 1167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.677671 1158 server_base.cc:1034] running on GCE node
W20260806 06:45:21.677544 1163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.678269 1158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:21.678556 1158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:21.679720 1158 hybrid_clock.cc:648] HybridClock initialized: now 1785998721679707 us; error 24 us; skew 500 ppm
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:21.682232 1158 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:45:21.682653 1158 webserver.cc:460] Webserver started at http://127.14.51.67:44359/ using document root <none> and password file <none>
I20260806 06:45:21.682896 1158 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:21.682972 1158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:21.683180 1158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:21.684190 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "7e179c640b3b8f69507bdefa89257f7e"
server_key_iv: "18f146ea1a9e8516776d82230c3e8303"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.684556 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "7e179c640b3b8f69507bdefa89257f7e"
server_key_iv: "18f146ea1a9e8516776d82230c3e8303"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.685873 1158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:21.686594 1174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.686764 1158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:21.686867 1158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "7e179c640b3b8f69507bdefa89257f7e"
server_key_iv: "18f146ea1a9e8516776d82230c3e8303"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.686950 1158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:21.695012 1158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:21.697293 1158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:21.697468 1158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:21.697714 1158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:21.698058 1158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:21.698118 1158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.698156 1158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:21.698182 1158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.704116 1158 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:43587
I20260806 06:45:21.704162 1294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:43587 every 8 connection(s)
I20260806 06:45:21.704516 1158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:45:21 dist-test-slave-1xrh krb5kdc[754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:21.713407 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.705239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:35153 (local address 127.14.51.126:44777)
0806 06:45:21.705305 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.705307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.705530 (+ 223us) server_negotiation.cc:407] Connection header received
0806 06:45:21.706036 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.706037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.706050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.706099 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:21.707249 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.707981 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.708886 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.709069 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.709116 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.711101 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:21.711103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:21.711103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:21.711108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:21.712299 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.712662 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.712663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.712663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.712709 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.712918 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.712919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.712919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.713012 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:21.713062 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.713296 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.713351 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:45:21.713624 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.705244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44777 (local address 127.14.51.67:35153)
0806 06:45:21.705404 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.705437 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.705912 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.706162 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.706166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.706860 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:21.707123 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.707129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.708071 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.708072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.708809 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.708811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.709142 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.709143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.709240 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.709721 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:21.709729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:21.711018 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:21.712375 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.712380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.712382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.712594 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.712772 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:21.712773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:21.712773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:21.712862 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:21.713073 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:21.713077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:21.713185 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.713270 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.713369 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260806 06:45:21.713801 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1158
I20260806 06:45:21.713896 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:21.714078 14541 external_mini_cluster.cc:1428] Setting key 543db64e2111a5437a51f4d0a30f5554
I20260806 06:45:21.714237 1295 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44777
I20260806 06:45:21.714324 1295 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:21.714504 1295 heartbeater.cc:511] Master 127.14.51.126:44777 requested a full tablet report, sending...
I20260806 06:45:21.714869 804 ts_manager.cc:194] Registered new tserver with Master: d9b1678bb9ee4ff89a28d7aa9a50b964 (127.14.51.67:43587)
I20260806 06:45:21.715325 804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:35153
I20260806 06:45:21.725955 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:21.733287 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.727346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38920 (local address 127.14.51.126:44777)
0806 06:45:21.727416 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.727417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.727478 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:45:21.727522 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.727523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.727531 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.727576 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:21.728077 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.728532 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.729240 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.729407 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.729454 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.730944 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:21.730945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:21.730945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:21.730950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:21.732192 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.732555 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.732556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.732556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.732603 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:21.732806 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:21.732807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:21.732807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:21.732919 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:21.732994 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.733118 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.733167 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:45:21.734984 804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:21.736505 804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:21.743599 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.738791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42151 (local address 127.0.0.1:49742)
0806 06:45:21.738951 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.739002 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.739065 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.739544 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.739547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.739549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:21.739625 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.739627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.740947 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.740948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.742141 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.742142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.743283 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.743284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.743348 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.743372 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.743418 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.743472 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":13,"thread_start_us":69,"threads_started":1}
I20260806 06:45:21.743845 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.738791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:43587 (local address 127.0.0.1:46820)
0806 06:45:21.739184 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.739194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.739237 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.739859 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.739860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.739862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:21.739915 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.739916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.741120 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.741121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.742053 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.742056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.743228 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.743229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.743352 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.743408 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.743466 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.743520 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260806 06:45:21.743921 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.738893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.14.51.66:42151)
0806 06:45:21.739365 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.739370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.739380 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:21.739424 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.739425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.739437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.739558 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:21.739717 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.740847 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.742267 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.743185 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.743571 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.743662 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.743713 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.743749 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":95,"threads_started":1}
I20260806 06:45:21.744230 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.738976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.14.51.67:43587)
0806 06:45:21.739695 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.739699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.739709 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:21.739743 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.739746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.739762 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.739996 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:21.740061 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.741014 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.742148 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.743081 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.743226 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.743287 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.744107 (+ 820us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.744147 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":73,"threads_started":1}
I20260806 06:45:21.744902 1227 tablet_service.cc:1467] Processing CreateTablet for tablet 64d9722e63584cc18a02e92793f90fcb (DEFAULT_TABLE table=test-table [id=48037e8e5ddc4adf95ce35120b64be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:21.745272 1227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64d9722e63584cc18a02e92793f90fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:21.745303 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.738978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:46073 (local address 127.0.0.1:53098)
0806 06:45:21.739270 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.739280 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.739322 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.739712 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.739713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.739716 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:21.739777 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.739778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.740789 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.740790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.742140 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.742142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.745028 (+ 2886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.745029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.745124 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.745130 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.745182 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.745228 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":35,"thread_start_us":77,"threads_started":1}
I20260806 06:45:21.745596 1084 tablet_service.cc:1467] Processing CreateTablet for tablet 64d9722e63584cc18a02e92793f90fcb (DEFAULT_TABLE table=test-table [id=48037e8e5ddc4adf95ce35120b64be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:21.745898 1084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64d9722e63584cc18a02e92793f90fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:21.747812 1312 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Bootstrap starting.
I20260806 06:45:21.748243 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.739293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.14.51.65:46073)
0806 06:45:21.739453 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.739458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.739468 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:21.739510 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.739512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.739526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.739628 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:21.739859 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.740686 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.742236 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.743998 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.744224 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.748076 (+ 3852us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.748111 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.748162 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260806 06:45:21.748468 1312 tablet_bootstrap.cc:654] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:21.748728 1313 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Bootstrap starting.
I20260806 06:45:21.748836 942 tablet_service.cc:1467] Processing CreateTablet for tablet 64d9722e63584cc18a02e92793f90fcb (DEFAULT_TABLE table=test-table [id=48037e8e5ddc4adf95ce35120b64be73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:21.749065 1312 log.cc:824] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:21.749172 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64d9722e63584cc18a02e92793f90fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:21.749396 1313 tablet_bootstrap.cc:654] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:21.749850 1313 log.cc:824] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:21.750552 1313 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: No bootstrap required, opened a new log
I20260806 06:45:21.750636 1313 ts_tablet_manager.cc:1397] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:21.751595 1312 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: No bootstrap required, opened a new log
I20260806 06:45:21.751633 1316 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: Bootstrap starting.
I20260806 06:45:21.751664 1312 ts_tablet_manager.cc:1397] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 06:45:21.752266 1316 tablet_bootstrap.cc:654] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:21.752355 1313 raft_consensus.cc:348] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.752556 1313 raft_consensus.cc:374] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:21.752600 1313 raft_consensus.cc:729] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577212ed1c0c4c31b981ef8f05e37b51, State: Initialized, Role: FOLLOWER
I20260806 06:45:21.752715 1313 consensus_queue.cc:260] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.752827 1316 log.cc:824] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:21.752990 1313 ts_tablet_manager.cc:1428] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:45:21.753183 1153 heartbeater.cc:503] Master 127.14.51.126:44777 was elected leader, sending a full tablet report...
I20260806 06:45:21.753692 1316 tablet_bootstrap.cc:492] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: No bootstrap required, opened a new log
I20260806 06:45:21.753764 1316 ts_tablet_manager.cc:1397] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:21.753748 1312 raft_consensus.cc:348] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.753911 1312 raft_consensus.cc:374] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:21.753944 1312 raft_consensus.cc:729] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b1678bb9ee4ff89a28d7aa9a50b964, State: Initialized, Role: FOLLOWER
I20260806 06:45:21.754057 1312 consensus_queue.cc:260] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.754308 1312 ts_tablet_manager.cc:1428] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:21.754462 1295 heartbeater.cc:503] Master 127.14.51.126:44777 was elected leader, sending a full tablet report...
I20260806 06:45:21.755846 1316 raft_consensus.cc:348] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.755966 1316 raft_consensus.cc:374] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:21.755986 1316 raft_consensus.cc:729] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4965d268972e4850b421db83d0385483, State: Initialized, Role: FOLLOWER
I20260806 06:45:21.756093 1316 consensus_queue.cc:260] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.756362 1316 ts_tablet_manager.cc:1428] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:21.756362 1010 heartbeater.cc:503] Master 127.14.51.126:44777 was elected leader, sending a full tablet report...
W20260806 06:45:21.805316 1154 tablet.cc:2369] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:21.814991 1321 raft_consensus.cc:482] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:21.815193 1321 raft_consensus.cc:504] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.815575 1321 leader_election.cc:288] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 577212ed1c0c4c31b981ef8f05e37b51 (127.14.51.66:42151), 4965d268972e4850b421db83d0385483 (127.14.51.65:46073)
I20260806 06:45:21.818584 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.815872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:38359 (local address 127.14.51.65:46073)
0806 06:45:21.815970 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.815978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.816078 (+ 100us) server_negotiation.cc:407] Connection header received
0806 06:45:21.816103 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.816104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.816117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.816176 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:21.816379 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.816854 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.817833 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.818212 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.818267 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.818273 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.818468 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.818515 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:45:21.818600 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.815752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:46073 (local address 127.14.51.67:38359)
0806 06:45:21.815958 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.815970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.816050 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.816200 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.816201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.816202 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:21.816299 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.816301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.816945 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.816946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.817728 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.817730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.818307 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.818308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.818388 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.818391 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.818434 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.818465 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":11,"thread_start_us":137,"threads_started":1}
I20260806 06:45:21.818658 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.815872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46293 (local address 127.14.51.66:42151)
0806 06:45:21.815954 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.815956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.815966 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:21.815997 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.815999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.816011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.816071 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:21.816266 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.816785 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.817838 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.818218 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.818318 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.818326 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.818541 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.818584 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:45:21.818610 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.815752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:42151 (local address 127.14.51.67:46293)
0806 06:45:21.815809 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.815821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:21.815879 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:21.816097 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:21.816098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:21.816100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:21.816176 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.816177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.816888 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.816890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.817729 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:21.817732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.818340 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.818341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:21.818415 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.818443 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 06:45:21.818501 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:45:21.818539 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":51}
I20260806 06:45:21.818948 962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64d9722e63584cc18a02e92793f90fcb" candidate_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4965d268972e4850b421db83d0385483" is_pre_election: true
I20260806 06:45:21.819108 962 raft_consensus.cc:2454] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b1678bb9ee4ff89a28d7aa9a50b964 in term 0.
I20260806 06:45:21.819146 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64d9722e63584cc18a02e92793f90fcb" candidate_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577212ed1c0c4c31b981ef8f05e37b51" is_pre_election: true
I20260806 06:45:21.819331 1105 raft_consensus.cc:2454] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b1678bb9ee4ff89a28d7aa9a50b964 in term 0.
I20260806 06:45:21.819355 1178 leader_election.cc:302] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4965d268972e4850b421db83d0385483, d9b1678bb9ee4ff89a28d7aa9a50b964; no voters:
I20260806 06:45:21.819465 1321 raft_consensus.cc:2781] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:21.819530 1321 raft_consensus.cc:482] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:21.819554 1321 raft_consensus.cc:3037] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:21.820289 1321 raft_consensus.cc:504] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.820429 1321 leader_election.cc:288] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [CANDIDATE]: Term 1 election: Requested vote from peers 577212ed1c0c4c31b981ef8f05e37b51 (127.14.51.66:42151), 4965d268972e4850b421db83d0385483 (127.14.51.65:46073)
I20260806 06:45:21.820632 962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64d9722e63584cc18a02e92793f90fcb" candidate_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4965d268972e4850b421db83d0385483"
I20260806 06:45:21.820631 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64d9722e63584cc18a02e92793f90fcb" candidate_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577212ed1c0c4c31b981ef8f05e37b51"
I20260806 06:45:21.820709 962 raft_consensus.cc:3037] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:21.820731 1105 raft_consensus.cc:3037] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:21.821413 962 raft_consensus.cc:2454] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b1678bb9ee4ff89a28d7aa9a50b964 in term 1.
I20260806 06:45:21.821503 1105 raft_consensus.cc:2454] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b1678bb9ee4ff89a28d7aa9a50b964 in term 1.
I20260806 06:45:21.821614 1178 leader_election.cc:302] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4965d268972e4850b421db83d0385483, d9b1678bb9ee4ff89a28d7aa9a50b964; no voters:
I20260806 06:45:21.821698 1321 raft_consensus.cc:2781] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:21.821827 1321 raft_consensus.cc:686] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 1 LEADER]: Becoming Leader. State: Replica: d9b1678bb9ee4ff89a28d7aa9a50b964, State: Running, Role: LEADER
I20260806 06:45:21.821925 1321 consensus_queue.cc:237] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } }
I20260806 06:45:21.822751 804 catalog_manager.cc:5495] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b1678bb9ee4ff89a28d7aa9a50b964 (127.14.51.67). New cstate: current_term: 1 leader_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9b1678bb9ee4ff89a28d7aa9a50b964" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 43587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 } health_report { overall_health: UNKNOWN } } }
I20260806 06:45:21.835285 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.833152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.14.51.67:43587)
0806 06:45:21.833207 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:21.833209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:21.833216 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:21.833245 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:21.833246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:21.833264 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:21.833327 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:21.833454 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.833878 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.834568 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:21.834747 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:21.834793 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:21.834947 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:21.835023 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:21.835093 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:21.835218 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:45:21.835228 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:45:21.836951 962 raft_consensus.cc:1261] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 1 FOLLOWER]: Refusing update from remote peer d9b1678bb9ee4ff89a28d7aa9a50b964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:21.836956 1105 raft_consensus.cc:1261] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 1 FOLLOWER]: Refusing update from remote peer d9b1678bb9ee4ff89a28d7aa9a50b964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:21.837271 1321 consensus_queue.cc:1035] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4965d268972e4850b421db83d0385483" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:21.837442 1324 consensus_queue.cc:1035] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577212ed1c0c4c31b981ef8f05e37b51" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:21.840153 1328 mvcc.cc:205] Tried to move back new op lower bound from 7315450764641599488 to 7315450764583223296. Current Snapshot: MvccSnapshot[applied={T|T < 7315450764641599488}]
I20260806 06:45:21.840276 1329 mvcc.cc:205] Tried to move back new op lower bound from 7315450764641599488 to 7315450764583223296. Current Snapshot: MvccSnapshot[applied={T|T < 7315450764641599488 or (T in {7315450764641599488})}]
I20260806 06:45:21.842315 802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:21.842404 802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:21.843202 802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64d1935da5f6451e9624a7272def8e38: Sending DeleteTablet for 3 replicas of tablet 64d9722e63584cc18a02e92793f90fcb
I20260806 06:45:21.843551 1084 tablet_service.cc:1514] Processing DeleteTablet for tablet 64d9722e63584cc18a02e92793f90fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:49742
I20260806 06:45:21.843567 1227 tablet_service.cc:1514] Processing DeleteTablet for tablet 64d9722e63584cc18a02e92793f90fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:46820
I20260806 06:45:21.843652 942 tablet_service.cc:1514] Processing DeleteTablet for tablet 64d9722e63584cc18a02e92793f90fcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:21 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:53098
I20260806 06:45:21.843895 1339 tablet_replica.cc:330] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483: stopping tablet replica
I20260806 06:45:21.843909 1337 tablet_replica.cc:330] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: stopping tablet replica
I20260806 06:45:21.843977 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 870
I20260806 06:45:21.843992 1339 raft_consensus.cc:2229] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:21.843992 1337 raft_consensus.cc:2229] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:21.844084 1339 raft_consensus.cc:2258] T 64d9722e63584cc18a02e92793f90fcb P 4965d268972e4850b421db83d0385483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:21.844545 1338 tablet_replica.cc:330] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: stopping tablet replica
I20260806 06:45:21.844640 1338 raft_consensus.cc:2229] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:21.844738 1338 raft_consensus.cc:2258] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:21.844901 1337 raft_consensus.cc:2258] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:21.845383 1338 ts_tablet_manager.cc:1905] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:21.845523 1337 ts_tablet_manager.cc:1905] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:21.847164 1338 ts_tablet_manager.cc:1918] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:21.847239 1338 log.cc:1196] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/64d9722e63584cc18a02e92793f90fcb
I20260806 06:45:21.847548 1338 ts_tablet_manager.cc:1939] T 64d9722e63584cc18a02e92793f90fcb P 577212ed1c0c4c31b981ef8f05e37b51: Deleting consensus metadata
I20260806 06:45:21.847640 1337 ts_tablet_manager.cc:1918] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:21.847697 1337 log.cc:1196] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/64d9722e63584cc18a02e92793f90fcb
I20260806 06:45:21.847940 1337 ts_tablet_manager.cc:1939] T 64d9722e63584cc18a02e92793f90fcb P d9b1678bb9ee4ff89a28d7aa9a50b964: Deleting consensus metadata
I20260806 06:45:21.847931 788 catalog_manager.cc:4841] TS 577212ed1c0c4c31b981ef8f05e37b51 (127.14.51.66:42151): tablet 64d9722e63584cc18a02e92793f90fcb (table test-table [id=48037e8e5ddc4adf95ce35120b64be73]) successfully deleted
I20260806 06:45:21.848326 789 catalog_manager.cc:4841] TS d9b1678bb9ee4ff89a28d7aa9a50b964 (127.14.51.67:43587): tablet 64d9722e63584cc18a02e92793f90fcb (table test-table [id=48037e8e5ddc4adf95ce35120b64be73]) successfully deleted
W20260806 06:45:21.852523 789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:46073 (error 108)
I20260806 06:45:21.852783 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1015
I20260806 06:45:21.853612 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:21.852910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:46073 (local address 127.0.0.1:53104)
0806 06:45:21.853538 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:21.853576 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608}
W20260806 06:45:21.854212 789 catalog_manager.cc:4584] TS 4965d268972e4850b421db83d0385483 (127.14.51.65:46073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64d9722e63584cc18a02e92793f90fcb: Network error: Client connection negotiation failed: client connection to 127.14.51.65:46073: connect: Connection refused (error 111)
I20260806 06:45:21.858633 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1158
I20260806 06:45:21.864612 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 768
2026-08-06T06:45:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1970 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T06:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:21Z Disabled control of system clock
I20260806 06:45:21.892198 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:39089
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:39123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:39089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:21.970963 1346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:21.971256 1346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:21.971282 1346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:21.972446 1346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:21.972486 1346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:21.972544 1346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:21.972576 1346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:21.972594 1346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:21.973893 1346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:39089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:21.974151 1346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:21.974442 1346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:45:21.979305 1346 server_base.cc:1034] running on GCE node
W20260806 06:45:21.979277 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.979288 1353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:21.979285 1354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:21.979883 1346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:21.980181 1346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:21.981350 1346 hybrid_clock.cc:648] HybridClock initialized: now 1785998721981318 us; error 44 us; skew 500 ppm
I20260806 06:45:21.989625 1346 webserver.cc:460] Webserver started at http://127.14.51.126:39797/ using document root <none> and password file <none>
I20260806 06:45:21.989912 1346 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:21.989996 1346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:21.990166 1346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:21.991323 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "a72dff1d97484a3f98f347f285833f15"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "b339c601f53b1e4d500afd1b9f6d0c08"
server_key_iv: "254fe4ee6ce8c7821ae4281fc9a448ca"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.991698 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "a72dff1d97484a3f98f347f285833f15"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "b339c601f53b1e4d500afd1b9f6d0c08"
server_key_iv: "254fe4ee6ce8c7821ae4281fc9a448ca"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.993000 1346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:21.993750 1362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:21.993934 1346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:21.994046 1346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "a72dff1d97484a3f98f347f285833f15"
format_stamp: "Formatted at 2026-08-06 06:45:21 on dist-test-slave-1xrh"
server_key: "b339c601f53b1e4d500afd1b9f6d0c08"
server_key_iv: "254fe4ee6ce8c7821ae4281fc9a448ca"
server_key_version: "encryptionkey@0"
I20260806 06:45:21.994134 1346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:22.014258 1346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:22.014582 1346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:22.014750 1346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:22.018663 1346 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:39089
I20260806 06:45:22.018700 1416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:39089 every 8 connection(s)
I20260806 06:45:22.019025 1346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:22.020077 1417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:22.022353 1417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: Bootstrap starting.
I20260806 06:45:22.022933 1417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:22.023272 1417 log.cc:824] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:22.023955 1417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: No bootstrap required, opened a new log
I20260806 06:45:22.025485 1417 raft_consensus.cc:348] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } }
I20260806 06:45:22.025617 1417 raft_consensus.cc:374] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:22.025640 1417 raft_consensus.cc:729] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72dff1d97484a3f98f347f285833f15, State: Initialized, Role: FOLLOWER
I20260806 06:45:22.025818 1417 consensus_queue.cc:260] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } }
I20260806 06:45:22.025897 1417 raft_consensus.cc:388] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:22.025964 1417 raft_consensus.cc:482] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:22.026026 1417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:22.026444 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1346
I20260806 06:45:22.026532 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:22.026585 1417 raft_consensus.cc:504] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } }
I20260806 06:45:22.026713 1417 leader_election.cc:302] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a72dff1d97484a3f98f347f285833f15; no voters:
I20260806 06:45:22.026770 14541 external_mini_cluster.cc:1428] Setting key 9913ec2bdf1134677a20d731b5472622
I20260806 06:45:22.026925 1417 leader_election.cc:288] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:22.026973 1420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:22.027192 1420 raft_consensus.cc:686] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [term 1 LEADER]: Becoming Leader. State: Replica: a72dff1d97484a3f98f347f285833f15, State: Running, Role: LEADER
I20260806 06:45:22.027302 1417 sys_catalog.cc:564] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:22.027364 1420 consensus_queue.cc:237] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } }
I20260806 06:45:22.027772 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a72dff1d97484a3f98f347f285833f15. Latest consensus state: current_term: 1 leader_uuid: "a72dff1d97484a3f98f347f285833f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } } }
I20260806 06:45:22.027925 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:22.028179 1431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:22.028725 1431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:22.028147 1422 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a72dff1d97484a3f98f347f285833f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dff1d97484a3f98f347f285833f15" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 39089 } } }
I20260806 06:45:22.028898 1422 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:22.030385 1431 catalog_manager.cc:1347] Generated new cluster ID: 3f86e8f0e6434b46b386bbcb76c7a632
I20260806 06:45:22.030437 1431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:22.032745 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.027326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.14.51.126:39089)
0806 06:45:22.027520 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.027528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.027689 (+ 161us) server_negotiation.cc:407] Connection header received
0806 06:45:22.027837 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.027843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.027911 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.028045 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:22.028263 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.029783 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.030773 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.031241 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.031358 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.032089 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:22.032093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:22.032093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:22.032264 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:22.032295 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:22.032305 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:22.032384 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.032466 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.032554 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260806 06:45:22.035892 1431 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:22.036360 1431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:22.038892 1431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: Generated new TSK 0
I20260806 06:45:22.039075 1431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:22.042150 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--builtin_ntp_servers=127.14.51.84:39123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:22.121508 1441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:22.121729 1441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:22.121766 1441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:22.122910 1441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:22.122982 1441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:22.123027 1441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:22.124442 1441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:22.124706 1441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:22.124996 1441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:22.130877 1450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:22.130964 1448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:22.130889 1447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:22.131429 1441 server_base.cc:1034] running on GCE node
I20260806 06:45:22.131633 1441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:22.131902 1441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:22.133065 1441 hybrid_clock.cc:648] HybridClock initialized: now 1785998722133051 us; error 40 us; skew 500 ppm
I20260806 06:45:22.134292 1441 webserver.cc:460] Webserver started at http://127.14.51.65:45585/ using document root <none> and password file <none>
I20260806 06:45:22.134552 1441 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:22.134630 1441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:22.134790 1441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:22.135898 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "63cf59034d6b465eb6ac397d62266813"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "e25f7ffc870fbcf47d9776925d006d2d"
server_key_iv: "3550077f8ec395c7d3b390a77429916f"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.136262 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "63cf59034d6b465eb6ac397d62266813"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "e25f7ffc870fbcf47d9776925d006d2d"
server_key_iv: "3550077f8ec395c7d3b390a77429916f"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.137594 1441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:22.138370 1457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.138643 1441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:22.138737 1441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "63cf59034d6b465eb6ac397d62266813"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "e25f7ffc870fbcf47d9776925d006d2d"
server_key_iv: "3550077f8ec395c7d3b390a77429916f"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.138803 1441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:22.157317 1441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:22.157568 1441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:22.157668 1441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:22.157842 1441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:22.158111 1441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:22.158140 1441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.158161 1441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:22.158174 1441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.163678 1441 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:42611
I20260806 06:45:22.163774 1576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:42611 every 8 connection(s)
I20260806 06:45:22.164113 1441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:22.167291 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1441
I20260806 06:45:22.167412 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:22.167625 14541 external_mini_cluster.cc:1428] Setting key c87555d6ad2596de57bd5cb8772a4707
I20260806 06:45:22.168606 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--builtin_ntp_servers=127.14.51.84:39123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:45:22.169902 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.164832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:36227 (local address 127.14.51.126:39089)
0806 06:45:22.164878 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.164881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.165079 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:45:22.165617 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.165619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.165633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.165688 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:22.166356 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.166847 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.168343 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.168544 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.168627 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.169620 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:22.169622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:22.169622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:22.169626 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:22.169642 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:22.169646 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:22.169720 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.169799 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.169827 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:45:22.170107 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.164786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39089 (local address 127.14.51.65:36227)
0806 06:45:22.164953 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.165001 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.165504 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.165752 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.165757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.165766 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:22.166252 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.166256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.166959 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.166960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.168272 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.168275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.168649 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.168650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.168805 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.169435 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:22.169438 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:22.169470 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:22.169497 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:22.169720 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:22.169727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:22.169730 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.169800 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.169896 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260806 06:45:22.170614 1577 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39089
I20260806 06:45:22.170702 1577 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:22.170908 1577 heartbeater.cc:511] Master 127.14.51.126:39089 requested a full tablet report, sending...
I20260806 06:45:22.171412 1379 ts_manager.cc:194] Registered new tserver with Master: 63cf59034d6b465eb6ac397d62266813 (127.14.51.65:42611)
I20260806 06:45:22.172304 1379 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.65:36227
W20260806 06:45:22.249214 1580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:22.249416 1580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:22.249436 1580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:22.250507 1580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:22.250552 1580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:22.250576 1580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:22.251780 1580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:22.251974 1580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:22.252208 1580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:22.256981 1586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:22.256958 1587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:22.257138 1580 server_base.cc:1034] running on GCE node
W20260806 06:45:22.256958 1589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:22.257596 1580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:22.257851 1580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:22.258997 1580 hybrid_clock.cc:648] HybridClock initialized: now 1785998722258987 us; error 27 us; skew 500 ppm
I20260806 06:45:22.260344 1580 webserver.cc:460] Webserver started at http://127.14.51.66:45023/ using document root <none> and password file <none>
I20260806 06:45:22.260607 1580 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:22.260654 1580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:22.260828 1580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:22.261828 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "b80bb9397020401bbf2c8cbe3b56d8af"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "cd0f3f1a738bc1c700a3be89d866ee67"
server_key_iv: "340f9a2eb98eeaf19876a53deb81d1b3"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.262187 1580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "b80bb9397020401bbf2c8cbe3b56d8af"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "cd0f3f1a738bc1c700a3be89d866ee67"
server_key_iv: "340f9a2eb98eeaf19876a53deb81d1b3"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.263592 1580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:22.264346 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.264532 1580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:22.264624 1580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "b80bb9397020401bbf2c8cbe3b56d8af"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "cd0f3f1a738bc1c700a3be89d866ee67"
server_key_iv: "340f9a2eb98eeaf19876a53deb81d1b3"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.264698 1580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:22.292892 1580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:22.293207 1580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:22.293380 1580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:22.293649 1580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:22.294034 1580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:22.294070 1580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.294091 1580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:22.294104 1580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.300653 1580 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:34495
I20260806 06:45:22.300719 1714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:34495 every 8 connection(s)
I20260806 06:45:22.301031 1580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:22.304364 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1580
I20260806 06:45:22.304497 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:22.304713 14541 external_mini_cluster.cc:1428] Setting key e725153059a1ebed2a8994a3f24cc44d
I20260806 06:45:22.305789 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--builtin_ntp_servers=127.14.51.84:39123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:45:22.306598 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.301767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:35573 (local address 127.14.51.126:39089)
0806 06:45:22.301815 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.301817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.302060 (+ 243us) server_negotiation.cc:407] Connection header received
0806 06:45:22.302516 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.302517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.302526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.302584 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:22.303283 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.303764 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.304916 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.305186 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.305246 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.306239 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:22.306240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:22.306241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:22.306247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:22.306273 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:22.306278 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:22.306339 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.306435 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.306466 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:45:22.306689 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.301726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39089 (local address 127.14.51.66:35573)
0806 06:45:22.301916 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.301972 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.302427 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.302634 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.302637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.302644 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:22.303192 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.303198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.303894 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.303895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.304814 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.304817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.305281 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.305282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.305476 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.306088 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:22.306089 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:22.306123 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:22.306145 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:22.306370 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:22.306374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:22.306376 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.306422 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.306498 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260806 06:45:22.307696 1715 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39089
I20260806 06:45:22.307840 1715 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:22.308079 1715 heartbeater.cc:511] Master 127.14.51.126:39089 requested a full tablet report, sending...
I20260806 06:45:22.308534 1379 ts_manager.cc:194] Registered new tserver with Master: b80bb9397020401bbf2c8cbe3b56d8af (127.14.51.66:34495)
I20260806 06:45:22.308939 1379 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.66:35573
W20260806 06:45:22.389434 1718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:22.389617 1718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:22.389639 1718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:22.390743 1718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:22.390784 1718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:22.390810 1718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:22.392195 1718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:39123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:22.392441 1718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:22.392771 1718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:22.398327 1727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:22.398343 1724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:22.398371 1725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:22.398564 1718 server_base.cc:1034] running on GCE node
I20260806 06:45:22.398770 1718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:22.399277 1718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:22.400507 1718 hybrid_clock.cc:648] HybridClock initialized: now 1785998722400474 us; error 42 us; skew 500 ppm
I20260806 06:45:22.401917 1718 webserver.cc:460] Webserver started at http://127.14.51.67:44403/ using document root <none> and password file <none>
I20260806 06:45:22.402212 1718 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:22.402302 1718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:22.402468 1718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:22.403690 1718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "f653044bc7dc4d06920ce201aebe5116"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "514b7316d6b4c4d96a39fc6c931d23fd"
server_key_iv: "8395cf0c4d0aa4ad1047b78b92914a2c"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.404139 1718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "f653044bc7dc4d06920ce201aebe5116"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "514b7316d6b4c4d96a39fc6c931d23fd"
server_key_iv: "8395cf0c4d0aa4ad1047b78b92914a2c"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.405660 1718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:45:22.406476 1733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.406674 1718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:22.406783 1718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "f653044bc7dc4d06920ce201aebe5116"
format_stamp: "Formatted at 2026-08-06 06:45:22 on dist-test-slave-1xrh"
server_key: "514b7316d6b4c4d96a39fc6c931d23fd"
server_key_iv: "8395cf0c4d0aa4ad1047b78b92914a2c"
server_key_version: "encryptionkey@0"
I20260806 06:45:22.406926 1718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:22.427862 1718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:22.428138 1718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:22.428282 1718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:22.428500 1718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:22.428840 1718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:22.428900 1718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.428941 1718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:22.428966 1718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:22.434450 1718 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37523
I20260806 06:45:22.434710 1856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37523 every 8 connection(s)
I20260806 06:45:22.435093 1718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:22.441525 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.435979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:57993 (local address 127.14.51.126:39089)
0806 06:45:22.436024 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.436026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.436274 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:45:22.436759 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.436760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.436772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.436848 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:22.437755 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.438459 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.440061 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.440239 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.440296 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.441228 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:22.441229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:22.441230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:22.441234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:22.441248 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:22.441252 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:22.441294 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.441439 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.441466 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:45:22.441771 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.435930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:39089 (local address 127.14.51.67:57993)
0806 06:45:22.436135 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.436176 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.436662 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.436904 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.436907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.436915 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:22.437628 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.437636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.438590 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.438592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.439967 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.439971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.440340 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.440341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.440494 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.441051 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:22.441051 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:22.441089 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:22.441118 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:22.441326 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:22.441333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:22.441336 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.441437 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.441547 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260806 06:45:22.442246 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1718
I20260806 06:45:22.442373 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:22.442370 1857 heartbeater.cc:347] Connected to a master server at 127.14.51.126:39089
I20260806 06:45:22.442449 1857 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:22.442603 14541 external_mini_cluster.cc:1428] Setting key 7b61593cfc9eeef34013d646b93709d7
I20260806 06:45:22.442651 1857 heartbeater.cc:511] Master 127.14.51.126:39089 requested a full tablet report, sending...
I20260806 06:45:22.443151 1379 ts_manager.cc:194] Registered new tserver with Master: f653044bc7dc4d06920ce201aebe5116 (127.14.51.67:37523)
I20260806 06:45:22.443586 1379 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.67:57993
I20260806 06:45:22.443662 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:22.448107 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.445263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.14.51.126:39089)
0806 06:45:22.445329 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.445331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.445441 (+ 110us) server_negotiation.cc:407] Connection header received
0806 06:45:22.445512 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.445513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.445523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.445576 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:22.445755 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.446189 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.446954 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.447185 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.447256 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.447837 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:22.447838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:22.447838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:22.447842 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:22.447853 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:22.447864 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:22.447967 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.448027 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.448035 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:45:22.449591 1379 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:22.450974 1379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:22.456966 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37523 (local address 127.0.0.1:48438)
0806 06:45:22.453367 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.453377 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.453430 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.453839 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.453843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.453845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:22.453933 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.453935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.454734 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.454735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.455697 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.455699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.456666 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.456667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.456739 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.456752 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.456825 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.456891 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":4,"thread_start_us":84,"threads_started":1}
I20260806 06:45:22.457195 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.14.51.67:37523)
0806 06:45:22.453590 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.453595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.453603 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:45:22.453635 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.453637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.453655 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.453768 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:22.453998 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.454644 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.455812 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.456555 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.456642 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.456726 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.456837 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.456899 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260806 06:45:22.457921 1788 tablet_service.cc:1467] Processing CreateTablet for tablet 077f192849314b4aa80774c58e3e30a8 (DEFAULT_TABLE table=test-table [id=3855c989093d4ee5965b14e862e3e89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:22.458223 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.14.51.66:34495)
0806 06:45:22.454092 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.454095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.454105 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:22.454143 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.454145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.454159 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.454226 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:22.454406 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.455444 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.456743 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.457730 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.457795 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.457891 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.458022 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.458064 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":104,"threads_started":1}
I20260806 06:45:22.458339 1788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077f192849314b4aa80774c58e3e30a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:22.458443 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:34495 (local address 127.0.0.1:34852)
0806 06:45:22.453836 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.453846 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.453893 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.454301 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.454302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.454304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:22.454335 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.454336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.455529 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.455530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.456654 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.456657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.457865 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.457867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.457926 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.457931 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.457985 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.458021 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":80,"threads_started":1}
I20260806 06:45:22.458549 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42611 (local address 127.0.0.1:49546)
0806 06:45:22.453199 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.453251 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.453320 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.453998 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.453999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.454001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:22.454035 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.454036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.455371 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.455372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.456338 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.456341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.458331 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.458332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.458420 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.458423 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.458455 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.458494 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260806 06:45:22.458932 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.453557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.14.51.65:42611)
0806 06:45:22.453799 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.453804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.453813 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:22.453849 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.453851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.453865 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.454116 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:22.454177 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.455271 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.456427 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.458239 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.458673 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.458780 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.458805 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.458848 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":180,"threads_started":1}
I20260806 06:45:22.459398 1648 tablet_service.cc:1467] Processing CreateTablet for tablet 077f192849314b4aa80774c58e3e30a8 (DEFAULT_TABLE table=test-table [id=3855c989093d4ee5965b14e862e3e89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:22.459547 1509 tablet_service.cc:1467] Processing CreateTablet for tablet 077f192849314b4aa80774c58e3e30a8 (DEFAULT_TABLE table=test-table [id=3855c989093d4ee5965b14e862e3e89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:22.459748 1648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077f192849314b4aa80774c58e3e30a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:22.459899 1509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077f192849314b4aa80774c58e3e30a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:22.461591 1873 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Bootstrap starting.
I20260806 06:45:22.462240 1873 tablet_bootstrap.cc:654] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:22.462553 1873 log.cc:824] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:22.463205 1875 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Bootstrap starting.
I20260806 06:45:22.463361 1873 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: No bootstrap required, opened a new log
I20260806 06:45:22.463425 1873 ts_tablet_manager.cc:1397] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:22.463883 1875 tablet_bootstrap.cc:654] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:22.464221 1875 log.cc:824] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:22.464732 1877 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Bootstrap starting.
I20260806 06:45:22.465066 1875 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: No bootstrap required, opened a new log
I20260806 06:45:22.465155 1875 ts_tablet_manager.cc:1397] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:22.465355 1877 tablet_bootstrap.cc:654] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:22.465682 1877 log.cc:824] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:22.466198 1873 raft_consensus.cc:348] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.466370 1873 raft_consensus.cc:374] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:22.466409 1873 raft_consensus.cc:729] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f653044bc7dc4d06920ce201aebe5116, State: Initialized, Role: FOLLOWER
I20260806 06:45:22.466508 1877 tablet_bootstrap.cc:492] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: No bootstrap required, opened a new log
I20260806 06:45:22.466570 1877 ts_tablet_manager.cc:1397] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:22.466535 1873 consensus_queue.cc:260] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.466795 1873 ts_tablet_manager.cc:1428] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:45:22.466823 1857 heartbeater.cc:503] Master 127.14.51.126:39089 was elected leader, sending a full tablet report...
I20260806 06:45:22.467669 1875 raft_consensus.cc:348] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.467818 1875 raft_consensus.cc:374] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:22.467859 1875 raft_consensus.cc:729] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80bb9397020401bbf2c8cbe3b56d8af, State: Initialized, Role: FOLLOWER
I20260806 06:45:22.467971 1875 consensus_queue.cc:260] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.468272 1875 ts_tablet_manager.cc:1428] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:22.468273 1715 heartbeater.cc:503] Master 127.14.51.126:39089 was elected leader, sending a full tablet report...
I20260806 06:45:22.469412 1877 raft_consensus.cc:348] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.469532 1877 raft_consensus.cc:374] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:22.469551 1877 raft_consensus.cc:729] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63cf59034d6b465eb6ac397d62266813, State: Initialized, Role: FOLLOWER
I20260806 06:45:22.469652 1877 consensus_queue.cc:260] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.469856 1877 ts_tablet_manager.cc:1428] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:22.469972 1577 heartbeater.cc:503] Master 127.14.51.126:39089 was elected leader, sending a full tablet report...
W20260806 06:45:22.551579 1716 tablet.cc:2369] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:22.633854 1881 raft_consensus.cc:482] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:22.634007 1881 raft_consensus.cc:504] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.634425 1881 leader_election.cc:288] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63cf59034d6b465eb6ac397d62266813 (127.14.51.65:42611), f653044bc7dc4d06920ce201aebe5116 (127.14.51.67:37523)
I20260806 06:45:22.638269 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.634660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:45241 (local address 127.14.51.65:42611)
0806 06:45:22.634705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.634707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.634734 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:45:22.634766 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.634767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.634775 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.634829 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:22.635065 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.635893 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637364 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.637892 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637965 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.637974 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.638162 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.638202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:45:22.638269 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.634694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:44745 (local address 127.14.51.67:37523)
0806 06:45:22.634744 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.634746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.634894 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:45:22.634931 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.634932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.634939 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.634977 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:22.635178 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.635891 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637364 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.637892 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637965 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.637974 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.638162 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.638202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:45:22.638279 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.634630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37523 (local address 127.14.51.66:44745)
0806 06:45:22.634773 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.634781 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.634857 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.635013 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.635014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.635015 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:22.635082 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.635083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.635981 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.635981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.637264 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.637266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.637994 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.638061 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.638065 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.638122 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.638176 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":72,"thread_start_us":100,"threads_started":1}
I20260806 06:45:22.638279 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.634561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42611 (local address 127.14.51.66:45241)
0806 06:45:22.634621 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.634634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:22.634681 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:22.634860 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:22.634861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:22.634863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:22.634938 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.634940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.635986 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.635987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.637264 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:22.637266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.637988 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.637989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:22.638061 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.638065 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:22.638122 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:45:22.638178 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":42}
I20260806 06:45:22.638895 1529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077f192849314b4aa80774c58e3e30a8" candidate_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63cf59034d6b465eb6ac397d62266813" is_pre_election: true
I20260806 06:45:22.638907 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077f192849314b4aa80774c58e3e30a8" candidate_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f653044bc7dc4d06920ce201aebe5116" is_pre_election: true
I20260806 06:45:22.639066 1529 raft_consensus.cc:2454] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80bb9397020401bbf2c8cbe3b56d8af in term 0.
I20260806 06:45:22.639078 1810 raft_consensus.cc:2454] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80bb9397020401bbf2c8cbe3b56d8af in term 0.
I20260806 06:45:22.639467 1599 leader_election.cc:302] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80bb9397020401bbf2c8cbe3b56d8af, f653044bc7dc4d06920ce201aebe5116; no voters:
I20260806 06:45:22.639635 1881 raft_consensus.cc:2781] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:22.639683 1881 raft_consensus.cc:482] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:22.639717 1881 raft_consensus.cc:3037] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:22.640637 1881 raft_consensus.cc:504] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.640795 1881 leader_election.cc:288] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [CANDIDATE]: Term 1 election: Requested vote from peers 63cf59034d6b465eb6ac397d62266813 (127.14.51.65:42611), f653044bc7dc4d06920ce201aebe5116 (127.14.51.67:37523)
I20260806 06:45:22.640916 1529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077f192849314b4aa80774c58e3e30a8" candidate_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63cf59034d6b465eb6ac397d62266813"
I20260806 06:45:22.640933 1810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077f192849314b4aa80774c58e3e30a8" candidate_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f653044bc7dc4d06920ce201aebe5116"
I20260806 06:45:22.641023 1529 raft_consensus.cc:3037] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:22.641019 1810 raft_consensus.cc:3037] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:22.641734 1810 raft_consensus.cc:2454] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80bb9397020401bbf2c8cbe3b56d8af in term 1.
I20260806 06:45:22.641808 1529 raft_consensus.cc:2454] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80bb9397020401bbf2c8cbe3b56d8af in term 1.
I20260806 06:45:22.641978 1600 leader_election.cc:302] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63cf59034d6b465eb6ac397d62266813, b80bb9397020401bbf2c8cbe3b56d8af; no voters:
I20260806 06:45:22.642093 1881 raft_consensus.cc:2781] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:22.642266 1881 raft_consensus.cc:686] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 1 LEADER]: Becoming Leader. State: Replica: b80bb9397020401bbf2c8cbe3b56d8af, State: Running, Role: LEADER
I20260806 06:45:22.642373 1881 consensus_queue.cc:237] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } }
I20260806 06:45:22.643226 1379 catalog_manager.cc:5495] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af reported cstate change: term changed from 0 to 1, leader changed from <none> to b80bb9397020401bbf2c8cbe3b56d8af (127.14.51.66). New cstate: current_term: 1 leader_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80bb9397020401bbf2c8cbe3b56d8af" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 34495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 } health_report { overall_health: UNKNOWN } } }
W20260806 06:45:22.664799 1578 tablet.cc:2369] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:22.682766 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.680389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.14.51.66:34495)
0806 06:45:22.680454 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:22.680456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:22.680468 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:45:22.680534 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:22.680534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:22.680554 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:22.680621 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:22.680769 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.681249 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.681998 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:22.682188 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:22.682252 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:22.682388 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:22.682473 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:22.682543 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:22.682598 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:45:22.682608 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:45:22.684793 1529 raft_consensus.cc:1261] T 077f192849314b4aa80774c58e3e30a8 P 63cf59034d6b465eb6ac397d62266813 [term 1 FOLLOWER]: Refusing update from remote peer b80bb9397020401bbf2c8cbe3b56d8af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:22.685112 1810 raft_consensus.cc:1261] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 1 FOLLOWER]: Refusing update from remote peer b80bb9397020401bbf2c8cbe3b56d8af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:22.685257 1881 consensus_queue.cc:1035] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "63cf59034d6b465eb6ac397d62266813" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:22.685565 1881 consensus_queue.cc:1035] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f653044bc7dc4d06920ce201aebe5116" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:45:22.685670 1858 tablet.cc:2369] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:22.688987 1889 mvcc.cc:205] Tried to move back new op lower bound from 7315450768114159616 to 7315450767943835648. Current Snapshot: MvccSnapshot[applied={T|T < 7315450768114159616 or (T in {7315450768114159616})}]
I20260806 06:45:22.690330 1377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:22.690412 1377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:22.691236 1377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a72dff1d97484a3f98f347f285833f15: Sending DeleteTablet for 3 replicas of tablet 077f192849314b4aa80774c58e3e30a8
I20260806 06:45:22.691617 1509 tablet_service.cc:1514] Processing DeleteTablet for tablet 077f192849314b4aa80774c58e3e30a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:22 UTC) from {username='slave'} at 127.0.0.1:49546
I20260806 06:45:22.691622 1648 tablet_service.cc:1514] Processing DeleteTablet for tablet 077f192849314b4aa80774c58e3e30a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:22 UTC) from {username='slave'} at 127.0.0.1:34852
I20260806 06:45:22.691851 1788 tablet_service.cc:1514] Processing DeleteTablet for tablet 077f192849314b4aa80774c58e3e30a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:22 UTC) from {username='slave'} at 127.0.0.1:48438
I20260806 06:45:22.691949 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1441
I20260806 06:45:22.693156 1900 tablet_replica.cc:330] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: stopping tablet replica
I20260806 06:45:22.693208 1898 tablet_replica.cc:330] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: stopping tablet replica
I20260806 06:45:22.693267 1900 raft_consensus.cc:2229] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:22.693292 1898 raft_consensus.cc:2229] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:22.693357 1900 raft_consensus.cc:2258] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:22.693370 1898 raft_consensus.cc:2258] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:22.693794 1898 ts_tablet_manager.cc:1905] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:22.694299 1900 ts_tablet_manager.cc:1905] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:22.695816 1898 ts_tablet_manager.cc:1918] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:22.695888 1898 log.cc:1196] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/077f192849314b4aa80774c58e3e30a8
I20260806 06:45:22.696174 1898 ts_tablet_manager.cc:1939] T 077f192849314b4aa80774c58e3e30a8 P b80bb9397020401bbf2c8cbe3b56d8af: Deleting consensus metadata
I20260806 06:45:22.696282 1900 ts_tablet_manager.cc:1918] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:22.696341 1900 log.cc:1196] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/077f192849314b4aa80774c58e3e30a8
I20260806 06:45:22.696579 1366 catalog_manager.cc:4841] TS b80bb9397020401bbf2c8cbe3b56d8af (127.14.51.66:34495): tablet 077f192849314b4aa80774c58e3e30a8 (table test-table [id=3855c989093d4ee5965b14e862e3e89d]) successfully deleted
I20260806 06:45:22.696676 1900 ts_tablet_manager.cc:1939] T 077f192849314b4aa80774c58e3e30a8 P f653044bc7dc4d06920ce201aebe5116: Deleting consensus metadata
I20260806 06:45:22.697058 1365 catalog_manager.cc:4841] TS f653044bc7dc4d06920ce201aebe5116 (127.14.51.67:37523): tablet 077f192849314b4aa80774c58e3e30a8 (table test-table [id=3855c989093d4ee5965b14e862e3e89d]) successfully deleted
W20260806 06:45:22.699993 1366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:42611 (error 108)
I20260806 06:45:22.700173 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1580
I20260806 06:45:22.702313 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:22.702134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42611 (local address 127.0.0.1:49562)
0806 06:45:22.702206 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:22.702251 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 06:45:22.702445 1366 catalog_manager.cc:4584] TS 63cf59034d6b465eb6ac397d62266813 (127.14.51.65:42611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 077f192849314b4aa80774c58e3e30a8: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42611: connect: Connection refused (error 111)
I20260806 06:45:22.705590 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1718
I20260806 06:45:22.710601 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1346
2026-08-06T06:45:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (845 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:22 dist-test-slave-1xrh krb5kdc[1904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:22 dist-test-slave-1xrh krb5kdc[1904](info): set up 2 sockets
Aug 06 06:45:22 dist-test-slave-1xrh krb5kdc[1904](info): commencing operation
krb5kdc: starting...
W20260806 06:45:23.775444 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:23 dist-test-slave-1xrh krb5kdc[1904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:23.893821 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33513
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:23.973662 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:23.973948 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:23.974051 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:23.975641 1918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:23.975682 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:23.975695 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:23.975708 1918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:23.975719 1918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:23.977099 1918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:23.977276 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:23.977524 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:23.982539 1925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:23.982630 1918 server_base.cc:1034] running on GCE node
W20260806 06:45:23.982723 1924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:23.982539 1927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:23.983093 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:23.983403 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:23.984551 1918 hybrid_clock.cc:648] HybridClock initialized: now 1785998723984538 us; error 46 us; skew 500 ppm
Aug 06 06:45:23 dist-test-slave-1xrh krb5kdc[1904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998723, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:23.986963 1918 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:23.987372 1918 webserver.cc:460] Webserver started at http://127.14.51.126:34597/ using document root <none> and password file <none>
I20260806 06:45:23.987583 1918 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:23.987623 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:23.987709 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:23.988653 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "09f4c8bdcb1542a2b6e4314997c99948"
format_stamp: "Formatted at 2026-08-06 06:45:23 on dist-test-slave-1xrh"
server_key: "96710d1dbd0dcc44e99152a75e8aa709"
server_key_iv: "77cdc3380358bafbb329b463f5544c02"
server_key_version: "encryptionkey@0"
I20260806 06:45:23.988957 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "09f4c8bdcb1542a2b6e4314997c99948"
format_stamp: "Formatted at 2026-08-06 06:45:23 on dist-test-slave-1xrh"
server_key: "96710d1dbd0dcc44e99152a75e8aa709"
server_key_iv: "77cdc3380358bafbb329b463f5544c02"
server_key_version: "encryptionkey@0"
I20260806 06:45:23.990192 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:23.990883 1934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:23.991037 1918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:23.991158 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "09f4c8bdcb1542a2b6e4314997c99948"
format_stamp: "Formatted at 2026-08-06 06:45:23 on dist-test-slave-1xrh"
server_key: "96710d1dbd0dcc44e99152a75e8aa709"
server_key_iv: "77cdc3380358bafbb329b463f5544c02"
server_key_version: "encryptionkey@0"
I20260806 06:45:23.991228 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:24.001446 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:24.003162 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:24.003302 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:24.007071 1918 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:33513
I20260806 06:45:24.007117 1990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:33513 every 8 connection(s)
I20260806 06:45:24.007511 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:24.007978 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 1918
I20260806 06:45:24.008093 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:24.008291 14541 external_mini_cluster.cc:1428] Setting key bc5b27379727e66ec3bb788d74a08d23
I20260806 06:45:24.008385 1991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:24.011493 1991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: Bootstrap starting.
I20260806 06:45:24.012108 1991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:24.012458 1991 log.cc:824] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:24.013232 1991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: No bootstrap required, opened a new log
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:24.015478 1991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } }
I20260806 06:45:24.015647 1991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:24.015717 1991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f4c8bdcb1542a2b6e4314997c99948, State: Initialized, Role: FOLLOWER
I20260806 06:45:24.015843 1991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } }
I20260806 06:45:24.015914 1991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:24.015936 1991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:24.016033 1991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:24.016613 1991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } }
I20260806 06:45:24.016728 1991 leader_election.cc:302] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09f4c8bdcb1542a2b6e4314997c99948; no voters:
I20260806 06:45:24.016907 1991 leader_election.cc:288] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:24.016939 1996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:24.017084 1996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [term 1 LEADER]: Becoming Leader. State: Replica: 09f4c8bdcb1542a2b6e4314997c99948, State: Running, Role: LEADER
I20260806 06:45:24.017202 1991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:24.017242 1996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } }
I20260806 06:45:24.017674 1996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09f4c8bdcb1542a2b6e4314997c99948. Latest consensus state: current_term: 1 leader_uuid: "09f4c8bdcb1542a2b6e4314997c99948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } } }
I20260806 06:45:24.017678 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09f4c8bdcb1542a2b6e4314997c99948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f4c8bdcb1542a2b6e4314997c99948" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33513 } } }
I20260806 06:45:24.017882 1996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:24.017894 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:24.018110 2005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:24.018682 2005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:24.019775 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.008775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.14.51.126:33513)
0806 06:45:24.008940 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.008949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.008980 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:24.009485 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.009492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.009555 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.009688 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.010283 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.011558 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.012366 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.012910 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.013018 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.015677 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.015681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.015681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:24.015695 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.017813 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.018240 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.018243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.018243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.018344 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.018572 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.018573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.018573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.018869 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.019418 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.019482 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.019581 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260806 06:45:24.020287 2005 catalog_manager.cc:1347] Generated new cluster ID: d799f88b63d046959ab077edc7702788
I20260806 06:45:24.020330 2005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:24.042815 2005 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:24.043548 2005 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:24.049923 2005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: Generated new TSK 0
I20260806 06:45:24.050117 2005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.51.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
Entry for principal kudu/127.14.51.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab.
I20260806 06:45:24.086654 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:24.166175 2017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:24.166369 2017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:24.166394 2017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:24.167569 2017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:24.167613 2017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:24.167639 2017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:24.169070 2017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.65.keytab
--principal=kudu/127.14.51.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:24.169306 2017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:24.169544 2017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:24.174515 2022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:24.174490 2025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:24.174491 2023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.174729 2017 server_base.cc:1034] running on GCE node
I20260806 06:45:24.175067 2017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:24.175334 2017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:24.176478 2017 hybrid_clock.cc:648] HybridClock initialized: now 1785998724176458 us; error 34 us; skew 500 ppm
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:24.179001 2017 init.cc:377] Logged in from keytab as kudu/127.14.51.65@KRBTEST.COM (short username kudu)
I20260806 06:45:24.179421 2017 webserver.cc:460] Webserver started at http://127.14.51.65:37835/ using document root <none> and password file <none>
I20260806 06:45:24.179592 2017 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:24.179630 2017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:24.179709 2017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:24.180605 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "df749c4d0a964ee984712c2af8690fe6"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "eaa6bc190953c3220fc77a1172b20dae"
server_key_iv: "9179052bfc57e6f883f3342a926773e1"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.180881 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "df749c4d0a964ee984712c2af8690fe6"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "eaa6bc190953c3220fc77a1172b20dae"
server_key_iv: "9179052bfc57e6f883f3342a926773e1"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.182114 2017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:24.182750 2032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.182950 2017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:24.183070 2017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "df749c4d0a964ee984712c2af8690fe6"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "eaa6bc190953c3220fc77a1172b20dae"
server_key_iv: "9179052bfc57e6f883f3342a926773e1"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.183169 2017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:24.197994 2017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:24.199728 2017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:24.199826 2017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:24.199981 2017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:24.200215 2017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:24.200245 2017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.200265 2017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:24.200278 2017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.205757 2017 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:34655
I20260806 06:45:24.205856 2149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:34655 every 8 connection(s)
I20260806 06:45:24.206126 2017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:24.211491 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2017
I20260806 06:45:24.211642 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:24.211822 14541 external_mini_cluster.cc:1428] Setting key c08c96332379e90825ed503b58982784
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.65@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:24.215397 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.206858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:52993 (local address 127.14.51.126:33513)
0806 06:45:24.206902 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.206904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.207081 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:45:24.207599 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.207600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.207609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.207682 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.208566 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.209064 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.210025 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.210215 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.210264 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.212341 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.212342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.212342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:24.212349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.214044 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.214401 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.214403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.214403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.214468 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.214686 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.214687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.214687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.214834 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.214887 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.215161 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.215254 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:45:24.215430 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.206802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33513 (local address 127.14.51.65:52993)
0806 06:45:24.206956 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.206995 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.207498 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.207730 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.207748 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.208301 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:24.208479 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.208483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.209147 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.209147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.209944 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.209946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.210289 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.210290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.210384 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.210823 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:24.210831 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:24.212258 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:24.214128 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.214134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.214136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.214326 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.214536 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.214536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.214536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.214622 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.214891 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:24.214895 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:24.214990 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.215076 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.215277 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260806 06:45:24.215905 2150 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33513
I20260806 06:45:24.216003 2150 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:24.216187 2150 heartbeater.cc:511] Master 127.14.51.126:33513 requested a full tablet report, sending...
I20260806 06:45:24.216696 1951 ts_manager.cc:194] Registered new tserver with Master: df749c4d0a964ee984712c2af8690fe6 (127.14.51.65:34655)
I20260806 06:45:24.217344 1951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.65@KRBTEST.COM'} at 127.14.51.65:52993
WARNING: no policy specified for kudu/127.14.51.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
Entry for principal kudu/127.14.51.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab.
I20260806 06:45:24.237615 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:24.315819 2155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:24.316010 2155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:24.316032 2155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:24.317299 2155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:24.317342 2155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:24.317390 2155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:24.318950 2155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.66.keytab
--principal=kudu/127.14.51.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:24.319191 2155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:24.319429 2155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:24.324360 2160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:24.324339 2161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.324455 2155 server_base.cc:1034] running on GCE node
W20260806 06:45:24.324338 2163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.324765 2155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:24.325026 2155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:24.326182 2155 hybrid_clock.cc:648] HybridClock initialized: now 1785998724326170 us; error 26 us; skew 500 ppm
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:24.328665 2155 init.cc:377] Logged in from keytab as kudu/127.14.51.66@KRBTEST.COM (short username kudu)
I20260806 06:45:24.329077 2155 webserver.cc:460] Webserver started at http://127.14.51.66:36843/ using document root <none> and password file <none>
I20260806 06:45:24.329301 2155 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:24.329372 2155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:24.329530 2155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:24.330596 2155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "3e5057ee510c4fadae50157c986425cc"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "7aa18bfef54e6fe9b32d2270512e2e65"
server_key_iv: "be507a22029da8802e609bf4e66fe6d4"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.330984 2155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "3e5057ee510c4fadae50157c986425cc"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "7aa18bfef54e6fe9b32d2270512e2e65"
server_key_iv: "be507a22029da8802e609bf4e66fe6d4"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.332459 2155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:24.333161 2170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.333329 2155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.333425 2155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "3e5057ee510c4fadae50157c986425cc"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "7aa18bfef54e6fe9b32d2270512e2e65"
server_key_iv: "be507a22029da8802e609bf4e66fe6d4"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.333509 2155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:24.351967 2155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:24.353703 2155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:24.353861 2155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:24.354110 2155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:24.354401 2155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:24.354457 2155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.354498 2155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:24.354521 2155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.360036 2155 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:40495
I20260806 06:45:24.360074 2288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:40495 every 8 connection(s)
I20260806 06:45:24.360399 2155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:24.361953 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2155
I20260806 06:45:24.362075 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:24.362264 14541 external_mini_cluster.cc:1428] Setting key 508ba1d4df6445c39907085a7b04044f
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.66@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.67@KRBTEST.COM; defaulting to no policy
I20260806 06:45:24.370822 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.361124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:33283 (local address 127.14.51.126:33513)
0806 06:45:24.361212 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.361215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.361383 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:45:24.361851 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.361853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.361867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.361947 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.363111 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.363935 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.365259 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.365443 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.365505 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.367918 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.367919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.367920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:24.367927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.369418 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.369856 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.369857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.369857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.369917 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.370172 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.370173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.370173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.370291 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.370371 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.370643 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.370723 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:45:24.371028 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.361095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33513 (local address 127.14.51.66:33283)
0806 06:45:24.361258 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.361291 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.361756 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.361999 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.362002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.362759 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:24.363018 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.363023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.364029 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.364030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.365167 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.365170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.365536 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.365537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.365656 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.366180 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:24.366187 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:24.367815 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:24.369540 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.369546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.369548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.369767 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.369993 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.369993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.369994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.370102 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.370362 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:24.370369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:24.370507 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.370610 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.370767 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":106,"threads_started":1}
I20260806 06:45:24.371497 2289 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33513
I20260806 06:45:24.371579 2289 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:24.371750 2289 heartbeater.cc:511] Master 127.14.51.126:33513 requested a full tablet report, sending...
I20260806 06:45:24.372149 1951 ts_manager.cc:194] Registered new tserver with Master: 3e5057ee510c4fadae50157c986425cc (127.14.51.66:40495)
I20260806 06:45:24.372637 1951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.66@KRBTEST.COM'} at 127.14.51.66:33283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
Entry for principal kudu/127.14.51.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab.
I20260806 06:45:24.387702 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:24.467665 2294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:24.467855 2294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:24.467880 2294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:24.469002 2294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:24.469043 2294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:24.469094 2294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:24.470546 2294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.67.keytab
--principal=kudu/127.14.51.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:33513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:24.470765 2294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:24.470997 2294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:24.475924 2299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:24.475935 2300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.475962 2294 server_base.cc:1034] running on GCE node
W20260806 06:45:24.475935 2302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.476343 2294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:24.476583 2294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:24.477747 2294 hybrid_clock.cc:648] HybridClock initialized: now 1785998724477740 us; error 39 us; skew 500 ppm
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:24.480115 2294 init.cc:377] Logged in from keytab as kudu/127.14.51.67@KRBTEST.COM (short username kudu)
I20260806 06:45:24.480525 2294 webserver.cc:460] Webserver started at http://127.14.51.67:46297/ using document root <none> and password file <none>
I20260806 06:45:24.480715 2294 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:24.480751 2294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:24.480829 2294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:24.481824 2294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "53b8ee3c9e9047d092e1774ea129a0f0"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "75eae570e288e6ed3f8e56a5fd07ea30"
server_key_iv: "80b6992868aae675116aac560f606ee3"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.482118 2294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "53b8ee3c9e9047d092e1774ea129a0f0"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "75eae570e288e6ed3f8e56a5fd07ea30"
server_key_iv: "80b6992868aae675116aac560f606ee3"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.483376 2294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:24.484225 2309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.484403 2294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.484493 2294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "53b8ee3c9e9047d092e1774ea129a0f0"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "75eae570e288e6ed3f8e56a5fd07ea30"
server_key_iv: "80b6992868aae675116aac560f606ee3"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.484565 2294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:24.508916 2294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:24.511276 2294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:24.511446 2294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:24.511711 2294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:24.512025 2294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:24.512085 2294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.512126 2294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:24.512156 2294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.517997 2294 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:37409
I20260806 06:45:24.518040 2431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:37409 every 8 connection(s)
I20260806 06:45:24.518391 2294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:24.522265 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2294
I20260806 06:45:24.522359 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:24.522509 14541 external_mini_cluster.cc:1428] Setting key 5fc0cf5ac8a2ccc715a47c8fd72dc01a
Aug 06 06:45:24 dist-test-slave-1xrh krb5kdc[1904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998724, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.67@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:24.527699 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.519168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:46787 (local address 127.14.51.126:33513)
0806 06:45:24.519228 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.519230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.519446 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:45:24.520077 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.520078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.520089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.520142 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.521097 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.521587 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.522835 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.523174 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.523241 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.525208 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.525209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.525209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:24.525215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.526588 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.526912 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.526913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.526913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.526961 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.527196 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.527196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.527197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.527294 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.527345 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.527569 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.527626 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:45:24.527859 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.519125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33513 (local address 127.14.51.67:46787)
0806 06:45:24.519315 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.519348 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.519976 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.520190 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.520194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.520776 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:24.521020 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.521025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.521665 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.521666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.522748 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.522751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.523254 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.523255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.523403 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.523842 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:24.523850 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:24.525131 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:24.526656 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.526661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.526662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.526848 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.527021 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:24.527022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:24.527022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:24.527109 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:24.527351 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:24.527355 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:24.527455 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.527544 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.527665 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260806 06:45:24.528267 2432 heartbeater.cc:347] Connected to a master server at 127.14.51.126:33513
I20260806 06:45:24.528365 2432 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:24.528528 2432 heartbeater.cc:511] Master 127.14.51.126:33513 requested a full tablet report, sending...
I20260806 06:45:24.528878 1951 ts_manager.cc:194] Registered new tserver with Master: 53b8ee3c9e9047d092e1774ea129a0f0 (127.14.51.67:37409)
I20260806 06:45:24.529246 1951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.51.67@KRBTEST.COM'} at 127.14.51.67:46787
I20260806 06:45:24.534195 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:24.540881 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.535453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.14.51.126:33513)
0806 06:45:24.535504 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.535506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.535630 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:45:24.535666 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.535667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.535674 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.535719 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.536307 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.536742 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.537508 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.537677 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.537728 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.538719 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.538720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.538720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:24.538726 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.540092 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.540351 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.540351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.540351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.540392 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:24.540531 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:24.540531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:24.540532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:24.540636 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.540699 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.540777 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.540828 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:45:24.542363 1951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:24.544034 1951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:24.550251 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.14.51.67:37409)
0806 06:45:24.546266 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.546271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.546383 (+ 112us) server_negotiation.cc:407] Connection header received
0806 06:45:24.546478 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.546480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.546495 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.546569 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:24.546857 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.547923 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.549223 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.549910 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.549986 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.550050 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.550140 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.550184 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260806 06:45:24.550253 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37409 (local address 127.0.0.1:39694)
0806 06:45:24.546285 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.546323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.546400 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.546644 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.546647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.546649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:24.546707 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.546709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.548060 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.548062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.549130 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.549132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.549999 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.550000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.550066 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.550075 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.550127 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.550160 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":32,"thread_start_us":53,"threads_started":1}
I20260806 06:45:24.550686 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34655 (local address 127.0.0.1:49128)
0806 06:45:24.546619 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.546629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.546688 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.546971 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.546972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.546974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:24.547054 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.547056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.547934 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.547935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.549312 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.549315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.550210 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.550211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.550276 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.550280 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.550559 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.550621 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":65,"thread_start_us":45,"threads_started":1}
I20260806 06:45:24.550985 2362 tablet_service.cc:1467] Processing CreateTablet for tablet 3db89a2b5218464aa6a00a03696327c5 (DEFAULT_TABLE table=test-table [id=10c11a8eef984ad799f44aa07ebf366b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:24.551340 2362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db89a2b5218464aa6a00a03696327c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:24.551358 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.14.51.65:34655)
0806 06:45:24.546594 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.546598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.546673 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:45:24.546764 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.546766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.546783 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.546883 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:24.547148 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.547818 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.549404 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.550126 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.550192 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.550243 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.551217 (+ 974us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.551265 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260806 06:45:24.551424 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:40495 (local address 127.0.0.1:59546)
0806 06:45:24.546502 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.546521 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.546564 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.546834 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.546835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.546837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:24.546879 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.546881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.548137 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.548138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.549312 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.549315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.551117 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.551118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.551248 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.551253 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.551310 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.551350 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":5,"thread_start_us":46,"threads_started":1}
I20260806 06:45:24.551741 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.546217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.14.51.66:40495)
0806 06:45:24.546388 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.546391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.546619 (+ 228us) server_negotiation.cc:407] Connection header received
0806 06:45:24.546662 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.546664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.546678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.546751 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:24.546957 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.548029 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.550279 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.551005 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.551095 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.551170 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.551606 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.551655 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260806 06:45:24.551955 2082 tablet_service.cc:1467] Processing CreateTablet for tablet 3db89a2b5218464aa6a00a03696327c5 (DEFAULT_TABLE table=test-table [id=10c11a8eef984ad799f44aa07ebf366b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:24.552300 2082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db89a2b5218464aa6a00a03696327c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:24.552335 2221 tablet_service.cc:1467] Processing CreateTablet for tablet 3db89a2b5218464aa6a00a03696327c5 (DEFAULT_TABLE table=test-table [id=10c11a8eef984ad799f44aa07ebf366b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:24.552634 2221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db89a2b5218464aa6a00a03696327c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:24.553884 2446 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Bootstrap starting.
I20260806 06:45:24.554512 2446 tablet_bootstrap.cc:654] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:24.555063 2446 log.cc:824] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:24.555609 2448 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Bootstrap starting.
I20260806 06:45:24.556077 2449 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Bootstrap starting.
I20260806 06:45:24.556357 2446 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: No bootstrap required, opened a new log
I20260806 06:45:24.556440 2446 ts_tablet_manager.cc:1397] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:45:24.556650 2449 tablet_bootstrap.cc:654] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:24.557091 2449 log.cc:824] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:24.557838 2449 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: No bootstrap required, opened a new log
I20260806 06:45:24.557890 2449 ts_tablet_manager.cc:1397] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:24.558163 2446 raft_consensus.cc:348] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.558323 2446 raft_consensus.cc:374] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:24.558368 2446 raft_consensus.cc:729] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b8ee3c9e9047d092e1774ea129a0f0, State: Initialized, Role: FOLLOWER
I20260806 06:45:24.558490 2446 consensus_queue.cc:260] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.558769 2446 ts_tablet_manager.cc:1428] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:45:24.558795 2432 heartbeater.cc:503] Master 127.14.51.126:33513 was elected leader, sending a full tablet report...
I20260806 06:45:24.559075 2448 tablet_bootstrap.cc:654] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:24.559644 2448 log.cc:824] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:24.559620 2449 raft_consensus.cc:348] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.559785 2449 raft_consensus.cc:374] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:24.559839 2449 raft_consensus.cc:729] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5057ee510c4fadae50157c986425cc, State: Initialized, Role: FOLLOWER
I20260806 06:45:24.559967 2449 consensus_queue.cc:260] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.560259 2449 ts_tablet_manager.cc:1428] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:45:24.560282 2289 heartbeater.cc:503] Master 127.14.51.126:33513 was elected leader, sending a full tablet report...
I20260806 06:45:24.560446 2448 tablet_bootstrap.cc:492] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: No bootstrap required, opened a new log
I20260806 06:45:24.560541 2448 ts_tablet_manager.cc:1397] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 06:45:24.562222 2448 raft_consensus.cc:348] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.562368 2448 raft_consensus.cc:374] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:24.562413 2448 raft_consensus.cc:729] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df749c4d0a964ee984712c2af8690fe6, State: Initialized, Role: FOLLOWER
I20260806 06:45:24.562520 2448 consensus_queue.cc:260] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.562762 2448 ts_tablet_manager.cc:1428] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:45:24.562814 2150 heartbeater.cc:503] Master 127.14.51.126:33513 was elected leader, sending a full tablet report...
W20260806 06:45:24.610952 2290 tablet.cc:2369] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:45:24.707015 2151 tablet.cc:2369] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:24.755170 2454 raft_consensus.cc:482] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:24.755332 2454 raft_consensus.cc:504] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.755748 2454 leader_election.cc:288] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53b8ee3c9e9047d092e1774ea129a0f0 (127.14.51.67:37409), df749c4d0a964ee984712c2af8690fe6 (127.14.51.65:34655)
I20260806 06:45:24.759814 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.756004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:52375 (local address 127.14.51.67:37409)
0806 06:45:24.756279 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.756281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.756291 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:24.756350 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.756351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.756362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.756418 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:24.756661 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.757407 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.758870 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.759456 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.759527 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.759534 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.759703 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.759740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260806 06:45:24.759814 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.756390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:52853 (local address 127.14.51.65:34655)
0806 06:45:24.756437 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.756439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.756446 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:24.756471 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.756472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.756481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.756536 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:24.756682 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.757409 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.758867 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.759444 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.759524 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.759534 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.759703 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.759740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:45:24.759840 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.755941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:37409 (local address 127.14.51.66:52375)
0806 06:45:24.756225 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.756239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.756292 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.756439 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.756440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.756441 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:24.756552 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.756554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.757495 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.757496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.758772 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.758775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.759532 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.759533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.759619 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.759622 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.759684 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.759739 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":49}
I20260806 06:45:24.759846 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.756072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34655 (local address 127.14.51.66:52853)
0806 06:45:24.756259 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.756272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:24.756361 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:24.756548 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:24.756549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:24.756550 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:24.756609 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.756610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.757495 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.757496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.758772 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:24.758775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.759532 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.759533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:24.759617 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.759622 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:45:24.759684 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:45:24.759739 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":57,"thread_start_us":104,"threads_started":1}
I20260806 06:45:24.760262 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db89a2b5218464aa6a00a03696327c5" candidate_uuid: "3e5057ee510c4fadae50157c986425cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df749c4d0a964ee984712c2af8690fe6" is_pre_election: true
I20260806 06:45:24.760267 2384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db89a2b5218464aa6a00a03696327c5" candidate_uuid: "3e5057ee510c4fadae50157c986425cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" is_pre_election: true
I20260806 06:45:24.760376 2103 raft_consensus.cc:2454] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5057ee510c4fadae50157c986425cc in term 0.
I20260806 06:45:24.760397 2384 raft_consensus.cc:2454] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5057ee510c4fadae50157c986425cc in term 0.
I20260806 06:45:24.760576 2173 leader_election.cc:302] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5057ee510c4fadae50157c986425cc, df749c4d0a964ee984712c2af8690fe6; no voters:
I20260806 06:45:24.760779 2454 raft_consensus.cc:2781] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:24.760819 2454 raft_consensus.cc:482] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:24.760869 2454 raft_consensus.cc:3037] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:24.761574 2454 raft_consensus.cc:504] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.761703 2454 leader_election.cc:288] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [CANDIDATE]: Term 1 election: Requested vote from peers 53b8ee3c9e9047d092e1774ea129a0f0 (127.14.51.67:37409), df749c4d0a964ee984712c2af8690fe6 (127.14.51.65:34655)
I20260806 06:45:24.761909 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db89a2b5218464aa6a00a03696327c5" candidate_uuid: "3e5057ee510c4fadae50157c986425cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df749c4d0a964ee984712c2af8690fe6"
I20260806 06:45:24.761926 2384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db89a2b5218464aa6a00a03696327c5" candidate_uuid: "3e5057ee510c4fadae50157c986425cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53b8ee3c9e9047d092e1774ea129a0f0"
I20260806 06:45:24.761997 2103 raft_consensus.cc:3037] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:24.761991 2384 raft_consensus.cc:3037] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:24.762677 2384 raft_consensus.cc:2454] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5057ee510c4fadae50157c986425cc in term 1.
I20260806 06:45:24.762753 2103 raft_consensus.cc:2454] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5057ee510c4fadae50157c986425cc in term 1.
I20260806 06:45:24.762840 2173 leader_election.cc:302] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5057ee510c4fadae50157c986425cc, 53b8ee3c9e9047d092e1774ea129a0f0; no voters:
I20260806 06:45:24.762943 2454 raft_consensus.cc:2781] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:24.763098 2454 raft_consensus.cc:686] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 1 LEADER]: Becoming Leader. State: Replica: 3e5057ee510c4fadae50157c986425cc, State: Running, Role: LEADER
I20260806 06:45:24.763224 2454 consensus_queue.cc:237] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } }
I20260806 06:45:24.764093 1951 catalog_manager.cc:5495] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5057ee510c4fadae50157c986425cc (127.14.51.66). New cstate: current_term: 1 leader_uuid: "3e5057ee510c4fadae50157c986425cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5057ee510c4fadae50157c986425cc" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 } health_report { overall_health: UNKNOWN } } }
W20260806 06:45:24.768934 2433 tablet.cc:2369] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:24.774164 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.772047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.14.51.66:40495)
0806 06:45:24.772107 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.772109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.772116 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:24.772179 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.772180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.772198 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.772255 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:24.772381 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.772815 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.773501 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.773676 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.773723 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.773856 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:24.773946 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:24.774002 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.774040 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.774054 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:45:24.775838 2103 raft_consensus.cc:1261] T 3db89a2b5218464aa6a00a03696327c5 P df749c4d0a964ee984712c2af8690fe6 [term 1 FOLLOWER]: Refusing update from remote peer 3e5057ee510c4fadae50157c986425cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:24.775838 2384 raft_consensus.cc:1261] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 1 FOLLOWER]: Refusing update from remote peer 3e5057ee510c4fadae50157c986425cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:24.776170 2457 consensus_queue.cc:1035] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "df749c4d0a964ee984712c2af8690fe6" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:24.776300 2454 consensus_queue.cc:1035] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "53b8ee3c9e9047d092e1774ea129a0f0" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 37409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:24.778966 2462 mvcc.cc:205] Tried to move back new op lower bound from 7315450776679366656 to 7315450776630755328. Current Snapshot: MvccSnapshot[applied={T|T < 7315450776679366656}]
I20260806 06:45:24.780762 1951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:24.780848 1951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:24.781610 1951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09f4c8bdcb1542a2b6e4314997c99948: Sending DeleteTablet for 3 replicas of tablet 3db89a2b5218464aa6a00a03696327c5
I20260806 06:45:24.781931 2221 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db89a2b5218464aa6a00a03696327c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:24 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:59546
I20260806 06:45:24.781960 2362 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db89a2b5218464aa6a00a03696327c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:24 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:39694
I20260806 06:45:24.782200 2082 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db89a2b5218464aa6a00a03696327c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:24 UTC) from {username='kudu', principal='kudu/127.14.51.126@KRBTEST.COM'} at 127.0.0.1:49128
I20260806 06:45:24.782249 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2017
I20260806 06:45:24.783286 2470 tablet_replica.cc:330] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: stopping tablet replica
I20260806 06:45:24.783411 2470 raft_consensus.cc:2229] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:24.783533 2470 raft_consensus.cc:2258] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:24.784169 2470 ts_tablet_manager.cc:1905] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:24.786196 2469 tablet_replica.cc:330] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: stopping tablet replica
I20260806 06:45:24.786283 2470 ts_tablet_manager.cc:1918] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:24.786345 2469 raft_consensus.cc:2229] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:24.786425 2470 log.cc:1196] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/3db89a2b5218464aa6a00a03696327c5
I20260806 06:45:24.786532 2469 raft_consensus.cc:2258] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:24.786864 2470 ts_tablet_manager.cc:1939] T 3db89a2b5218464aa6a00a03696327c5 P 53b8ee3c9e9047d092e1774ea129a0f0: Deleting consensus metadata
I20260806 06:45:24.787063 2469 ts_tablet_manager.cc:1905] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:24.787360 1937 catalog_manager.cc:4841] TS 53b8ee3c9e9047d092e1774ea129a0f0 (127.14.51.67:37409): tablet 3db89a2b5218464aa6a00a03696327c5 (table test-table [id=10c11a8eef984ad799f44aa07ebf366b]) successfully deleted
I20260806 06:45:24.789134 2469 ts_tablet_manager.cc:1918] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:24.789209 2469 log.cc:1196] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/3db89a2b5218464aa6a00a03696327c5
I20260806 06:45:24.789464 2469 ts_tablet_manager.cc:1939] T 3db89a2b5218464aa6a00a03696327c5 P 3e5057ee510c4fadae50157c986425cc: Deleting consensus metadata
I20260806 06:45:24.789857 1936 catalog_manager.cc:4841] TS 3e5057ee510c4fadae50157c986425cc (127.14.51.66:40495): tablet 3db89a2b5218464aa6a00a03696327c5 (table test-table [id=10c11a8eef984ad799f44aa07ebf366b]) successfully deleted
W20260806 06:45:24.790469 1937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:34655 (error 108)
I20260806 06:45:24.790700 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2155
I20260806 06:45:24.792480 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.790865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:34655 (local address 127.0.0.1:49144)
0806 06:45:24.792395 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:24.792440 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:34655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1505}
W20260806 06:45:24.792593 1937 catalog_manager.cc:4584] TS df749c4d0a964ee984712c2af8690fe6 (127.14.51.65:34655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3db89a2b5218464aa6a00a03696327c5: Network error: Client connection negotiation failed: client connection to 127.14.51.65:34655: connect: Connection refused (error 111)
I20260806 06:45:24.796978 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2294
I20260806 06:45:24.802750 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 1918
2026-08-06T06:45:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T06:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:24Z Disabled control of system clock
I20260806 06:45:24.830632 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44925
--webserver_interface=127.14.51.126
--webserver_port=0
--builtin_ntp_servers=127.14.51.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:44925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:24.911058 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:24.911355 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:24.911393 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:24.912582 2476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:24.912654 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:24.912676 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:24.912698 2476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:24.912735 2476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:24.914047 2476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:44925
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:24.914350 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:24.914645 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:24.919426 2486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.919552 2476 server_base.cc:1034] running on GCE node
W20260806 06:45:24.919421 2483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:24.919647 2482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:24.919981 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:24.920212 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:24.921391 2476 hybrid_clock.cc:648] HybridClock initialized: now 1785998724921368 us; error 35 us; skew 500 ppm
I20260806 06:45:24.922600 2476 webserver.cc:460] Webserver started at http://127.14.51.126:38115/ using document root <none> and password file <none>
I20260806 06:45:24.922883 2476 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:24.922960 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:24.923166 2476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:24.924270 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "067358a63a8846dba2d08d96500d138c"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "ed4accf1e7f6877419f143b29c8e5969"
server_key_iv: "f31d7c1bda62d36ced4eca54cea93980"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.924630 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "067358a63a8846dba2d08d96500d138c"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "ed4accf1e7f6877419f143b29c8e5969"
server_key_iv: "f31d7c1bda62d36ced4eca54cea93980"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.925940 2476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:24.926666 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:24.926863 2476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:24.926968 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "067358a63a8846dba2d08d96500d138c"
format_stamp: "Formatted at 2026-08-06 06:45:24 on dist-test-slave-1xrh"
server_key: "ed4accf1e7f6877419f143b29c8e5969"
server_key_iv: "f31d7c1bda62d36ced4eca54cea93980"
server_key_version: "encryptionkey@0"
I20260806 06:45:24.927053 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:24.934724 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:24.934990 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:24.935197 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:24.938902 2476 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:44925
I20260806 06:45:24.938931 2547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:44925 every 8 connection(s)
I20260806 06:45:24.939289 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:24.940148 2548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:24.942232 2548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: Bootstrap starting.
I20260806 06:45:24.942795 2548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:24.943091 2548 log.cc:824] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:24.943763 2548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: No bootstrap required, opened a new log
I20260806 06:45:24.944734 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2476
I20260806 06:45:24.944820 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:24.945027 14541 external_mini_cluster.cc:1428] Setting key c760e6dbcddcad5e33db6998b6a47343
I20260806 06:45:24.945602 2548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } }
I20260806 06:45:24.945783 2548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:24.945820 2548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067358a63a8846dba2d08d96500d138c, State: Initialized, Role: FOLLOWER
I20260806 06:45:24.945958 2548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } }
I20260806 06:45:24.946034 2548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:24.946064 2548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:24.946102 2548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:24.947100 2548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } }
I20260806 06:45:24.947230 2548 leader_election.cc:302] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 067358a63a8846dba2d08d96500d138c; no voters:
I20260806 06:45:24.947378 2548 leader_election.cc:288] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:24.947427 2554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:24.947572 2554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [term 1 LEADER]: Becoming Leader. State: Replica: 067358a63a8846dba2d08d96500d138c, State: Running, Role: LEADER
I20260806 06:45:24.947604 2548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:24.947745 2554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } }
I20260806 06:45:24.948169 2555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "067358a63a8846dba2d08d96500d138c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } } }
I20260806 06:45:24.948261 2555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:24.948413 2556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 067358a63a8846dba2d08d96500d138c. Latest consensus state: current_term: 1 leader_uuid: "067358a63a8846dba2d08d96500d138c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067358a63a8846dba2d08d96500d138c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44925 } } }
I20260806 06:45:24.948472 2556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:24.948812 2562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:24.949327 2562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:24.951053 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:24.945582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.14.51.126:44925)
0806 06:45:24.945808 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:24.945816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:24.945848 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:45:24.945969 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:24.945976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:24.946054 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:24.946197 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:24.946442 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.947921 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.949013 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:24.949598 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:24.949723 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:24.950384 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:24.950387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:24.950387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:24.950584 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:24.950622 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:24.950635 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:24.950699 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:24.950779 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:45:24.950860 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260806 06:45:24.951339 2562 catalog_manager.cc:1347] Generated new cluster ID: 3617d8e097534cf08dd7e9c507b89c9b
I20260806 06:45:24.951409 2562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:24.971136 2562 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:24.971671 2562 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:24.978519 2562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: Generated new TSK 0
I20260806 06:45:24.978705 2562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:24.985965 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.65:0
--local_ip_for_outbound_sockets=127.14.51.65
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--builtin_ntp_servers=127.14.51.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:25.067415 2573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:25.067598 2573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:25.067619 2573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:25.068742 2573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:25.068780 2573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:25.068804 2573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.65
I20260806 06:45:25.070124 2573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.51.65
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.65
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:25.070381 2573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:25.070717 2573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:25.075624 2578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:25.075615 2579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:25.075618 2581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:25.075821 2573 server_base.cc:1034] running on GCE node
I20260806 06:45:25.076081 2573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:25.076305 2573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:25.077445 2573 hybrid_clock.cc:648] HybridClock initialized: now 1785998725077433 us; error 29 us; skew 500 ppm
I20260806 06:45:25.078835 2573 webserver.cc:460] Webserver started at http://127.14.51.65:37571/ using document root <none> and password file <none>
I20260806 06:45:25.079105 2573 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:25.079208 2573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:25.079346 2573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:25.080372 2573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data/instance:
uuid: "9733991191bd4391a490638b2643162e"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "13160973285310b88b53dc15002239c4"
server_key_iv: "f7ddfa326343ca60f73b594512d575c6"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.080724 2573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance:
uuid: "9733991191bd4391a490638b2643162e"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "13160973285310b88b53dc15002239c4"
server_key_iv: "f7ddfa326343ca60f73b594512d575c6"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.081998 2573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:25.082686 2587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.082877 2573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:25.082976 2573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal
uuid: "9733991191bd4391a490638b2643162e"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "13160973285310b88b53dc15002239c4"
server_key_iv: "f7ddfa326343ca60f73b594512d575c6"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.083076 2573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:25.097767 2573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:25.098042 2573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:25.098184 2573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:25.098399 2573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:25.098681 2573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:25.098733 2573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.098768 2573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:25.098812 2573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.104646 2573 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.65:42303
I20260806 06:45:25.104700 2709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.65:42303 every 8 connection(s)
I20260806 06:45:25.105011 2573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/data/info.pb
I20260806 06:45:25.109436 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.105683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:52847 (local address 127.14.51.126:44925)
0806 06:45:25.105740 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.105742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.105941 (+ 199us) server_negotiation.cc:407] Connection header received
0806 06:45:25.106336 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.106337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.106348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.106407 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:25.106977 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.107456 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.108284 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.108449 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.108494 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.109251 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:25.109252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:25.109252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:25.109256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:25.109265 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:25.109268 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:25.109303 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.109359 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.109380 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:45:25.109603 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.105643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44925 (local address 127.14.51.65:52847)
0806 06:45:25.105818 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.105859 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.106258 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.106450 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.106453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.106460 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:25.106904 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.106907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.107527 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.107527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.108213 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.108215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.108519 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.108519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.108611 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.109135 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:25.109135 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:25.109161 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:25.109182 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:25.109305 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:25.109307 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:25.109309 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.109351 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.109419 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260806 06:45:25.110162 2710 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44925
I20260806 06:45:25.110234 2710 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:25.110417 2710 heartbeater.cc:511] Master 127.14.51.126:44925 requested a full tablet report, sending...
I20260806 06:45:25.110915 2510 ts_manager.cc:194] Registered new tserver with Master: 9733991191bd4391a490638b2643162e (127.14.51.65:42303)
I20260806 06:45:25.111320 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2573
I20260806 06:45:25.111415 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-0/wal/instance
I20260806 06:45:25.111593 14541 external_mini_cluster.cc:1428] Setting key 393c235902793a92a179f63f2a0813ee
I20260806 06:45:25.111768 2510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.65:52847
I20260806 06:45:25.112718 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.66:0
--local_ip_for_outbound_sockets=127.14.51.66
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--builtin_ntp_servers=127.14.51.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:45:25.191886 2713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:25.192109 2713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:25.192150 2713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:25.193279 2713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:25.193347 2713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:25.193403 2713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.66
I20260806 06:45:25.194672 2713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.51.66
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.66
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:25.194909 2713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:25.195226 2713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:25.199731 2718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:25.199748 2721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:25.199775 2719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:25.200019 2713 server_base.cc:1034] running on GCE node
I20260806 06:45:25.200206 2713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:25.200377 2713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:25.201514 2713 hybrid_clock.cc:648] HybridClock initialized: now 1785998725201505 us; error 38 us; skew 500 ppm
I20260806 06:45:25.202701 2713 webserver.cc:460] Webserver started at http://127.14.51.66:33765/ using document root <none> and password file <none>
I20260806 06:45:25.202946 2713 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:25.202991 2713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:25.203197 2713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:25.204211 2713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data/instance:
uuid: "527934e4adad4692ad55e1e1eda28cd0"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "565ca3217a6e5347f54d33b7f33d99b6"
server_key_iv: "07c44bd8a114e3c1ac3df4904bb06c42"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.204566 2713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance:
uuid: "527934e4adad4692ad55e1e1eda28cd0"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "565ca3217a6e5347f54d33b7f33d99b6"
server_key_iv: "07c44bd8a114e3c1ac3df4904bb06c42"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.205857 2713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:25.206525 2727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.206689 2713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:25.206786 2713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal
uuid: "527934e4adad4692ad55e1e1eda28cd0"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "565ca3217a6e5347f54d33b7f33d99b6"
server_key_iv: "07c44bd8a114e3c1ac3df4904bb06c42"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.206888 2713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:25.225106 2713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:25.225387 2713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:25.225536 2713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:25.225761 2713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:25.226084 2713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:25.226142 2713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.226177 2713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:25.226203 2713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.232414 2713 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.66:45233
I20260806 06:45:25.232461 2847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.66:45233 every 8 connection(s)
I20260806 06:45:25.232764 2713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/data/info.pb
I20260806 06:45:25.237439 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2713
I20260806 06:45:25.237600 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal/instance
I20260806 06:45:25.237762 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.233498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.66:34455 (local address 127.14.51.126:44925)
0806 06:45:25.233569 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.233571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.233779 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:45:25.234213 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.234214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.234223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.234278 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:25.234898 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.235504 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.236451 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.236636 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.236687 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.237424 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:25.237426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:25.237426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:25.237431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:25.237448 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:25.237453 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:25.237514 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.237644 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.237673 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:45:25.237840 14541 external_mini_cluster.cc:1428] Setting key 7c76890b5044796ddf67199dd917b39c
I20260806 06:45:25.237946 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.233481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44925 (local address 127.14.51.66:34455)
0806 06:45:25.233657 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.233694 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.234135 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.234324 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.234327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.234334 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:25.234830 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.234834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.235618 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.235619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.236374 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.236376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.236712 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.236713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.236817 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.237277 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:25.237277 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:25.237303 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:25.237326 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:25.237552 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:25.237557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:25.237560 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.237625 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.237734 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260806 06:45:25.238459 2850 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44925
I20260806 06:45:25.238584 2850 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:25.238777 2850 heartbeater.cc:511] Master 127.14.51.126:44925 requested a full tablet report, sending...
I20260806 06:45:25.238835 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.51.67:0
--local_ip_for_outbound_sockets=127.14.51.67
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--builtin_ntp_servers=127.14.51.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:45:25.239197 2509 ts_manager.cc:194] Registered new tserver with Master: 527934e4adad4692ad55e1e1eda28cd0 (127.14.51.66:45233)
I20260806 06:45:25.239560 2509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.66:34455
W20260806 06:45:25.322220 2853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:25.322474 2853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:25.322512 2853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:25.324105 2853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:25.324172 2853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:25.324216 2853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.51.67
I20260806 06:45:25.326050 2853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.51.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.51.67
--webserver_port=0
--tserver_master_addrs=127.14.51.126:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.51.67
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:25.326296 2853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:25.326678 2853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:25.332152 2859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:25.332118 2862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:25.332192 2853 server_base.cc:1034] running on GCE node
W20260806 06:45:25.332118 2860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:25.332484 2853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:25.332665 2853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:25.333829 2853 hybrid_clock.cc:648] HybridClock initialized: now 1785998725333818 us; error 26 us; skew 500 ppm
I20260806 06:45:25.334970 2853 webserver.cc:460] Webserver started at http://127.14.51.67:40767/ using document root <none> and password file <none>
I20260806 06:45:25.335273 2853 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:25.335345 2853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:25.335522 2853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:25.336519 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data/instance:
uuid: "0a35da8513b744afba692af34986aead"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "f1fd4c8d0c8a15e2218080dbb0199980"
server_key_iv: "2ff7508824b02f930b51c229998d89f1"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.336875 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance:
uuid: "0a35da8513b744afba692af34986aead"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "f1fd4c8d0c8a15e2218080dbb0199980"
server_key_iv: "2ff7508824b02f930b51c229998d89f1"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.338121 2853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:25.338805 2868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.338982 2853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:25.339072 2853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal
uuid: "0a35da8513b744afba692af34986aead"
format_stamp: "Formatted at 2026-08-06 06:45:25 on dist-test-slave-1xrh"
server_key: "f1fd4c8d0c8a15e2218080dbb0199980"
server_key_iv: "2ff7508824b02f930b51c229998d89f1"
server_key_version: "encryptionkey@0"
I20260806 06:45:25.339182 2853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:25.362826 2853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:25.363173 2853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:25.363339 2853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:25.363579 2853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:45:25.363925 2853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:45:25.363983 2853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.364017 2853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:45:25.364044 2853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:25.369679 2853 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.67:46271
I20260806 06:45:25.369762 2992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.67:46271 every 8 connection(s)
I20260806 06:45:25.370091 2853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/data/info.pb
I20260806 06:45:25.375175 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.370861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.67:32963 (local address 127.14.51.126:44925)
0806 06:45:25.370919 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.370922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.371274 (+ 352us) server_negotiation.cc:407] Connection header received
0806 06:45:25.371758 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.371759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.371770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.371819 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:25.372450 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.372942 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.373904 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.374089 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.374141 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.374889 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:25.374890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:25.374890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:25.374895 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:25.374909 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:25.374912 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:25.374954 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.375038 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.375063 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:45:25.375398 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.370849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:44925 (local address 127.14.51.67:32963)
0806 06:45:25.371077 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.371112 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.371673 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.371877 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.371880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.371888 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:25.372381 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.372385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.373036 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.373036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.373826 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.373828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.374165 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.374165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.374269 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.374760 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:45:25.374760 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:25.374788 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:45:25.374810 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:25.374965 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:25.374968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:25.374970 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.375025 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.375102 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
I20260806 06:45:25.375895 2993 heartbeater.cc:347] Connected to a master server at 127.14.51.126:44925
I20260806 06:45:25.376015 2993 heartbeater.cc:464] Registering TS with master...
I20260806 06:45:25.376199 2993 heartbeater.cc:511] Master 127.14.51.126:44925 requested a full tablet report, sending...
I20260806 06:45:25.376585 2509 ts_manager.cc:194] Registered new tserver with Master: 0a35da8513b744afba692af34986aead (127.14.51.67:46271)
I20260806 06:45:25.376947 2509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.51.67:32963
I20260806 06:45:25.377756 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 2853
I20260806 06:45:25.377884 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal/instance
I20260806 06:45:25.378085 14541 external_mini_cluster.cc:1428] Setting key dbd766a726a03fc80baaaaf19a33b3aa
I20260806 06:45:25.378947 14541 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:45:25.383368 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.380381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.14.51.126:44925)
0806 06:45:25.380452 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.380454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.380572 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:45:25.380643 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.380644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.380657 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.380705 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:25.380864 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.381329 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.382094 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.382290 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.382364 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.383072 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:25.383073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:25.383074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:25.383078 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:25.383090 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:25.383093 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:25.383203 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.383266 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.383288 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:45:25.385035 2509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:45:25.386837 2509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:45:25.393105 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.389126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.14.51.67:46271)
0806 06:45:25.389290 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.389294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.389433 (+ 139us) server_negotiation.cc:407] Connection header received
0806 06:45:25.389477 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.389480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.389496 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.389590 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:25.389730 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.390809 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.391912 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392608 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392678 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.392766 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.392972 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.393017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260806 06:45:25.393148 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.388947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:46271 (local address 127.0.0.1:39252)
0806 06:45:25.389345 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.389355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.389415 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.389603 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.389605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.389607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:25.389651 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.389652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.390903 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.390904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.391825 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.391828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392807 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.392887 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.392890 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.392957 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.393022 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":114,"threads_started":1}
I20260806 06:45:25.393414 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.389002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.14.51.65:42303)
0806 06:45:25.389188 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.389193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.389205 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:45:25.389330 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.389332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.389350 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.389432 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:25.389946 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.390654 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392015 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392715 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392786 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.392867 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.393271 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.393317 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260806 06:45:25.393419 3006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.389319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.14.51.66:45233)
0806 06:45:25.389681 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.389685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.389694 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:25.389730 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.389732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.389746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.389821 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:25.390015 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.391071 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392260 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392729 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392823 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.392907 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.393268 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.393300 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":96,"threads_started":1}
I20260806 06:45:25.393545 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.389115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45233 (local address 127.0.0.1:58090)
0806 06:45:25.389504 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.389514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.389563 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.389815 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.389816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.389818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:25.389915 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.389917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.391175 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.391176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.392169 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.392170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392939 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.392996 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.393000 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.393054 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.393085 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":76,"threads_started":1}
I20260806 06:45:25.393165 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.388906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42303 (local address 127.0.0.1:51570)
0806 06:45:25.389114 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.389154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.389213 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.389655 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.389656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.389657 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:25.389861 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.389863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.390763 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.390764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.391926 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.391928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.392803 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.392805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.392878 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.392889 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.392963 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.393034 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":22,"thread_start_us":72,"threads_started":1}
I20260806 06:45:25.394040 2918 tablet_service.cc:1467] Processing CreateTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 (DEFAULT_TABLE table=test-table [id=20107b7fc1ad4b9e8122da8428e7f856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:25.394198 2777 tablet_service.cc:1467] Processing CreateTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 (DEFAULT_TABLE table=test-table [id=20107b7fc1ad4b9e8122da8428e7f856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:25.394405 2918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ae28a56d8d494390c9862c9d0bae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:25.394342 2638 tablet_service.cc:1467] Processing CreateTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 (DEFAULT_TABLE table=test-table [id=20107b7fc1ad4b9e8122da8428e7f856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:45:25.394459 2777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ae28a56d8d494390c9862c9d0bae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:25.394587 2638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ae28a56d8d494390c9862c9d0bae25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:25.397351 3007 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: Bootstrap starting.
I20260806 06:45:25.397718 3008 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Bootstrap starting.
I20260806 06:45:25.398049 3007 tablet_bootstrap.cc:654] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:25.398315 3008 tablet_bootstrap.cc:654] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:25.398366 3007 log.cc:824] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:25.398650 3008 log.cc:824] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:25.398670 3009 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Bootstrap starting.
I20260806 06:45:25.399178 3007 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: No bootstrap required, opened a new log
I20260806 06:45:25.399253 3007 ts_tablet_manager.cc:1397] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:45:25.399299 3009 tablet_bootstrap.cc:654] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:25.399394 3008 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: No bootstrap required, opened a new log
I20260806 06:45:25.399458 3008 ts_tablet_manager.cc:1397] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:25.399588 3009 log.cc:824] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:25.400399 3009 tablet_bootstrap.cc:492] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: No bootstrap required, opened a new log
I20260806 06:45:25.400465 3009 ts_tablet_manager.cc:1397] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:45:25.401486 3008 raft_consensus.cc:348] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.401644 3008 raft_consensus.cc:374] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:25.401688 3008 raft_consensus.cc:729] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527934e4adad4692ad55e1e1eda28cd0, State: Initialized, Role: FOLLOWER
I20260806 06:45:25.401798 3008 consensus_queue.cc:260] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.401953 3007 raft_consensus.cc:348] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.402076 3008 ts_tablet_manager.cc:1428] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:25.402108 3007 raft_consensus.cc:374] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:25.402151 3007 raft_consensus.cc:729] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9733991191bd4391a490638b2643162e, State: Initialized, Role: FOLLOWER
I20260806 06:45:25.402208 2850 heartbeater.cc:503] Master 127.14.51.126:44925 was elected leader, sending a full tablet report...
I20260806 06:45:25.402266 3007 consensus_queue.cc:260] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.402498 3007 ts_tablet_manager.cc:1428] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:25.402594 2710 heartbeater.cc:503] Master 127.14.51.126:44925 was elected leader, sending a full tablet report...
I20260806 06:45:25.402957 3009 raft_consensus.cc:348] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.403110 3009 raft_consensus.cc:374] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:25.403173 3009 raft_consensus.cc:729] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a35da8513b744afba692af34986aead, State: Initialized, Role: FOLLOWER
I20260806 06:45:25.403280 3009 consensus_queue.cc:260] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.403501 3009 ts_tablet_manager.cc:1428] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:45:25.403592 2993 heartbeater.cc:503] Master 127.14.51.126:44925 was elected leader, sending a full tablet report...
W20260806 06:45:25.483286 2851 tablet.cc:2369] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:45:25.558132 3014 raft_consensus.cc:482] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:45:25.558311 3014 raft_consensus.cc:504] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.558722 3014 leader_election.cc:288] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 527934e4adad4692ad55e1e1eda28cd0 (127.14.51.66:45233), 0a35da8513b744afba692af34986aead (127.14.51.67:46271)
I20260806 06:45:25.562539 3006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.558976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:51359 (local address 127.14.51.66:45233)
0806 06:45:25.559056 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.559059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.559303 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:45:25.559363 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.559364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.559371 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.559442 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:25.559590 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.560257 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.561607 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.562160 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.562232 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.562240 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.562424 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.562463 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:45:25.562532 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.558995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.51.65:42123 (local address 127.14.51.67:46271)
0806 06:45:25.559043 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.559045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.559056 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:45:25.559159 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.559160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.559170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.559239 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:45:25.559431 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.560195 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.561608 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.562156 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.562232 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.562240 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.562424 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.562463 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:45:25.562534 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.558915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.67:46271 (local address 127.14.51.65:42123)
0806 06:45:25.559000 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.559015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.559068 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.559271 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.559272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.559274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:25.559349 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.559351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.560276 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.560277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.561509 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.561512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.562250 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.562251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.562332 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.562339 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.562396 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.562440 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":12}
I20260806 06:45:25.562548 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.558917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.66:45233 (local address 127.14.51.65:51359)
0806 06:45:25.559245 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.559256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:25.559308 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:25.559462 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:25.559462 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:25.559464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:45:25.559521 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.559522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.560331 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.560331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.561519 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:25.561521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.562250 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.562251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:25.562332 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.562339 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:45:25.562396 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:45:25.562439 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":111,"thread_start_us":137,"threads_started":1}
I20260806 06:45:25.562882 2940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ae28a56d8d494390c9862c9d0bae25" candidate_uuid: "9733991191bd4391a490638b2643162e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a35da8513b744afba692af34986aead" is_pre_election: true
I20260806 06:45:25.563037 2940 raft_consensus.cc:2454] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9733991191bd4391a490638b2643162e in term 0.
I20260806 06:45:25.563079 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ae28a56d8d494390c9862c9d0bae25" candidate_uuid: "9733991191bd4391a490638b2643162e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527934e4adad4692ad55e1e1eda28cd0" is_pre_election: true
I20260806 06:45:25.563261 2798 raft_consensus.cc:2454] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9733991191bd4391a490638b2643162e in term 0.
I20260806 06:45:25.563303 2589 leader_election.cc:302] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a35da8513b744afba692af34986aead, 9733991191bd4391a490638b2643162e; no voters:
I20260806 06:45:25.563460 3014 raft_consensus.cc:2781] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:45:25.563534 3014 raft_consensus.cc:482] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:45:25.563552 3014 raft_consensus.cc:3037] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:25.564253 3014 raft_consensus.cc:504] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.564392 3014 leader_election.cc:288] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [CANDIDATE]: Term 1 election: Requested vote from peers 527934e4adad4692ad55e1e1eda28cd0 (127.14.51.66:45233), 0a35da8513b744afba692af34986aead (127.14.51.67:46271)
I20260806 06:45:25.564555 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ae28a56d8d494390c9862c9d0bae25" candidate_uuid: "9733991191bd4391a490638b2643162e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527934e4adad4692ad55e1e1eda28cd0"
I20260806 06:45:25.564589 2940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ae28a56d8d494390c9862c9d0bae25" candidate_uuid: "9733991191bd4391a490638b2643162e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a35da8513b744afba692af34986aead"
I20260806 06:45:25.564637 2798 raft_consensus.cc:3037] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:25.564669 2940 raft_consensus.cc:3037] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:25.565272 2798 raft_consensus.cc:2454] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9733991191bd4391a490638b2643162e in term 1.
I20260806 06:45:25.565478 2590 leader_election.cc:302] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 527934e4adad4692ad55e1e1eda28cd0, 9733991191bd4391a490638b2643162e; no voters:
I20260806 06:45:25.565541 2940 raft_consensus.cc:2454] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9733991191bd4391a490638b2643162e in term 1.
I20260806 06:45:25.565580 3014 raft_consensus.cc:2781] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:25.565725 3014 raft_consensus.cc:686] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 1 LEADER]: Becoming Leader. State: Replica: 9733991191bd4391a490638b2643162e, State: Running, Role: LEADER
I20260806 06:45:25.565809 3014 consensus_queue.cc:237] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } }
I20260806 06:45:25.566726 2510 catalog_manager.cc:5495] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9733991191bd4391a490638b2643162e (127.14.51.65). New cstate: current_term: 1 leader_uuid: "9733991191bd4391a490638b2643162e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9733991191bd4391a490638b2643162e" member_type: VOTER last_known_addr { host: "127.14.51.65" port: 42303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260806 06:45:25.572152 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.569794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.14.51.65:42303)
0806 06:45:25.569854 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:25.569856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:25.569866 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:25.569907 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:25.569908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:25.569932 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:25.570015 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:25.570140 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.570568 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.571387 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:25.571575 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:25.571650 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:25.571825 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:25.571915 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:25.572018 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:25.572036 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 06:45:25.572053 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:45:25.573658 2940 raft_consensus.cc:1261] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 1 FOLLOWER]: Refusing update from remote peer 9733991191bd4391a490638b2643162e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:25.573685 2798 raft_consensus.cc:1261] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 1 FOLLOWER]: Refusing update from remote peer 9733991191bd4391a490638b2643162e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:25.574000 3014 consensus_queue.cc:1035] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a35da8513b744afba692af34986aead" member_type: VOTER last_known_addr { host: "127.14.51.67" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:25.574136 3019 consensus_queue.cc:1035] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [LEADER]: Connected to new peer: Peer: permanent_uuid: "527934e4adad4692ad55e1e1eda28cd0" member_type: VOTER last_known_addr { host: "127.14.51.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:25.577093 3024 mvcc.cc:205] Tried to move back new op lower bound from 7315450779947520000 to 7315450779918372864. Current Snapshot: MvccSnapshot[applied={T|T < 7315450779947520000}]
I20260806 06:45:25.578969 2510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:25.579062 2510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:45:25.579756 2510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 067358a63a8846dba2d08d96500d138c: Sending DeleteTablet for 3 replicas of tablet 61ae28a56d8d494390c9862c9d0bae25
I20260806 06:45:25.580051 2638 tablet_service.cc:1514] Processing DeleteTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:25 UTC) from {username='slave'} at 127.0.0.1:51570
I20260806 06:45:25.580119 2777 tablet_service.cc:1514] Processing DeleteTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:25 UTC) from {username='slave'} at 127.0.0.1:58090
I20260806 06:45:25.580157 2918 tablet_service.cc:1514] Processing DeleteTablet for tablet 61ae28a56d8d494390c9862c9d0bae25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:45:25 UTC) from {username='slave'} at 127.0.0.1:39252
I20260806 06:45:25.580324 3033 tablet_replica.cc:330] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: stopping tablet replica
I20260806 06:45:25.580446 3033 raft_consensus.cc:2229] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:25.580469 3032 tablet_replica.cc:330] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e: stopping tablet replica
I20260806 06:45:25.580540 3033 raft_consensus.cc:2258] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:25.580564 3032 raft_consensus.cc:2229] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:45:25.580570 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2573
I20260806 06:45:25.580637 3032 raft_consensus.cc:2258] T 61ae28a56d8d494390c9862c9d0bae25 P 9733991191bd4391a490638b2643162e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:25.580976 3033 ts_tablet_manager.cc:1905] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:25.582916 3033 ts_tablet_manager.cc:1918] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:25.582985 3033 log.cc:1196] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-1/wal/wals/61ae28a56d8d494390c9862c9d0bae25
I20260806 06:45:25.583262 3033 ts_tablet_manager.cc:1939] T 61ae28a56d8d494390c9862c9d0bae25 P 527934e4adad4692ad55e1e1eda28cd0: Deleting consensus metadata
I20260806 06:45:25.583439 3034 tablet_replica.cc:330] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: stopping tablet replica
I20260806 06:45:25.583535 3034 raft_consensus.cc:2229] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:45:25.583634 3034 raft_consensus.cc:2258] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:45:25.584060 3034 ts_tablet_manager.cc:1905] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:45:25.584201 2495 catalog_manager.cc:4841] TS 527934e4adad4692ad55e1e1eda28cd0 (127.14.51.66:45233): tablet 61ae28a56d8d494390c9862c9d0bae25 (table test-table [id=20107b7fc1ad4b9e8122da8428e7f856]) successfully deleted
I20260806 06:45:25.585299 3034 ts_tablet_manager.cc:1918] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:45:25.585361 3034 log.cc:1196] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Deleting WAL directory at /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998482572191-14541-0/minicluster-data/ts-2/wal/wals/61ae28a56d8d494390c9862c9d0bae25
I20260806 06:45:25.585693 3034 ts_tablet_manager.cc:1939] T 61ae28a56d8d494390c9862c9d0bae25 P 0a35da8513b744afba692af34986aead: Deleting consensus metadata
I20260806 06:45:25.586095 2494 catalog_manager.cc:4841] TS 0a35da8513b744afba692af34986aead (127.14.51.67:46271): tablet 61ae28a56d8d494390c9862c9d0bae25 (table test-table [id=20107b7fc1ad4b9e8122da8428e7f856]) successfully deleted
W20260806 06:45:25.587148 2495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.51.65:42303 (error 108)
I20260806 06:45:25.587435 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2713
I20260806 06:45:25.587584 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:25.587451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.65:42303 (local address 127.0.0.1:51584)
0806 06:45:25.587512 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:25.587549 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 06:45:25.588088 2495 catalog_manager.cc:4584] TS 9733991191bd4391a490638b2643162e (127.14.51.65:42303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61ae28a56d8d494390c9862c9d0bae25: Network error: Client connection negotiation failed: client connection to 127.14.51.65:42303: connect: Connection refused (error 111)
I20260806 06:45:25.593600 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2853
I20260806 06:45:25.598124 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 2476
2026-08-06T06:45:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (794 ms)
[----------] 6 tests from EncryptionPolicyTest (8728 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:25 dist-test-slave-1xrh krb5kdc[3038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:25 dist-test-slave-1xrh krb5kdc[3038](info): set up 2 sockets
Aug 06 06:45:25 dist-test-slave-1xrh krb5kdc[3038](info): commencing operation
krb5kdc: starting...
W20260806 06:45:26.638705 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:26.753993 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:26.834264 3051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:26.834504 3051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:26.834542 3051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:26.835687 3051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:26.835763 3051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:26.835790 3051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:26.835822 3051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:26.835844 3051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:26.837270 3051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:26.837540 3051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:26.837764 3051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:26.842582 3056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:26.842566 3060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:26.842674 3051 server_base.cc:1034] running on GCE node
W20260806 06:45:26.842571 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:26.843067 3051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:26.843385 3051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:26.844566 3051 hybrid_clock.cc:648] HybridClock initialized: now 1785998726844561 us; error 37 us; skew 500 ppm
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:26.847079 3051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:26.847522 3051 webserver.cc:460] Webserver started at http://127.0.0.1:37587/ using document root <none> and password file <none>
I20260806 06:45:26.847780 3051 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:26.847852 3051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:26.848001 3051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:26.848966 3051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "1728cd6d6f2d4989957e04af2303d088"
format_stamp: "Formatted at 2026-08-06 06:45:26 on dist-test-slave-1xrh"
server_key: "3fbbefbb5df7e34bbcd02c75f144d8f3"
server_key_iv: "ce20b44238677989451579280e0ebcf9"
server_key_version: "encryptionkey@0"
I20260806 06:45:26.849424 3051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "1728cd6d6f2d4989957e04af2303d088"
format_stamp: "Formatted at 2026-08-06 06:45:26 on dist-test-slave-1xrh"
server_key: "3fbbefbb5df7e34bbcd02c75f144d8f3"
server_key_iv: "ce20b44238677989451579280e0ebcf9"
server_key_version: "encryptionkey@0"
I20260806 06:45:26.850721 3051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:26.851501 3068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:26.851677 3051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:26.851781 3051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "1728cd6d6f2d4989957e04af2303d088"
format_stamp: "Formatted at 2026-08-06 06:45:26 on dist-test-slave-1xrh"
server_key: "3fbbefbb5df7e34bbcd02c75f144d8f3"
server_key_iv: "ce20b44238677989451579280e0ebcf9"
server_key_version: "encryptionkey@0"
I20260806 06:45:26.851861 3051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:26.869488 3051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:26.871892 3051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:26.872068 3051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:26.875916 3051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43745
I20260806 06:45:26.875948 3123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43745 every 8 connection(s)
I20260806 06:45:26.876294 3051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:26.877099 3124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:26.878325 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3051
I20260806 06:45:26.878443 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:26.878635 14541 external_mini_cluster.cc:1428] Setting key 1591c59177ddc96196fa065fdb6ef2d9
I20260806 06:45:26.879729 3124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088: Bootstrap starting.
I20260806 06:45:26.880451 3124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:26.880793 3124 log.cc:824] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:26.881500 3124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088: No bootstrap required, opened a new log
I20260806 06:45:26.883832 3124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260806 06:45:26.884049 3124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:26.884095 3124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1728cd6d6f2d4989957e04af2303d088, State: Initialized, Role: FOLLOWER
I20260806 06:45:26.884227 3124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260806 06:45:26.884315 3124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:26.884347 3124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:26.884402 3124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:26.885130 3124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260806 06:45:26.885215 3124 leader_election.cc:302] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1728cd6d6f2d4989957e04af2303d088; no voters:
I20260806 06:45:26.885424 3124 leader_election.cc:288] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:26.885443 3129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:26.885573 3129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [term 1 LEADER]: Becoming Leader. State: Replica: 1728cd6d6f2d4989957e04af2303d088, State: Running, Role: LEADER
I20260806 06:45:26.885667 3129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260806 06:45:26.885703 3124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:45:26.885943 3129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1728cd6d6f2d4989957e04af2303d088. Latest consensus state: current_term: 1 leader_uuid: "1728cd6d6f2d4989957e04af2303d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } } }
I20260806 06:45:26.886013 3129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:26.885958 3130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1728cd6d6f2d4989957e04af2303d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1728cd6d6f2d4989957e04af2303d088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } } }
I20260806 06:45:26.886204 3130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:26.886221 3133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:26.886855 3133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:26.888695 3133 catalog_manager.cc:1347] Generated new cluster ID: eac30928bf394535bec35d60e890ee27
I20260806 06:45:26.888751 3133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:26.890154 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:26.879223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34559 (local address 127.0.0.1:43745)
0806 06:45:26.879396 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:26.879404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:26.879428 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:45:26.879871 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:26.879879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:26.879940 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:26.880126 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:26.880931 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:26.881870 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:26.882915 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:26.883362 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:26.883440 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:26.886279 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:26.886282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:26.886283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:26.886294 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:26.888453 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:26.888901 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:26.888905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:26.888905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:26.889029 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:26.889285 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:26.889286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:26.889287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:26.889592 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:26.889734 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:26.889860 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 06:45:26.889960 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260806 06:45:26.891176 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:26.896631 3133 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:26.897249 3133 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:26.901500 3133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1728cd6d6f2d4989957e04af2303d088: Generated new TSK 0
I20260806 06:45:26.901674 3133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:26.901975 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:26.892758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50977 (local address 127.0.0.1:43745)
0806 06:45:26.892819 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:26.892821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:26.895300 (+ 2479us) server_negotiation.cc:407] Connection header received
0806 06:45:26.895341 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:26.895342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:26.895352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:26.895404 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:26.895986 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:26.896530 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:26.897470 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:26.897798 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:26.897871 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:26.899104 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:26.899105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:26.899106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:26.899112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:26.900692 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:26.901096 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:26.901097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:26.901097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:26.901155 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:26.901397 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:26.901398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:26.901398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:26.901509 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:26.901581 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:26.901823 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 06:45:26.901889 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:45:26.904145 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3051
W20260806 06:45:26.910233 3036 connection.cc:535] client connection to 127.0.0.1:43745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43745): Connection reset by peer (error 104)
2026-08-06T06:45:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3157](info): set up 2 sockets
Aug 06 06:45:26 dist-test-slave-1xrh krb5kdc[3157](info): commencing operation
krb5kdc: starting...
W20260806 06:45:27.958541 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:28 dist-test-slave-1xrh krb5kdc[3157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:28.078001 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43285
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:28.157271 3170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:28.157492 3170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:28.157531 3170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:28.158600 3170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:28.158669 3170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:28.158695 3170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:28.158723 3170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:28.158754 3170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:28.160238 3170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43285
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:28.160491 3170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:28.160710 3170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:28.165499 3176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:28.165547 3178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:28.165549 3175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:28.165583 3170 server_base.cc:1034] running on GCE node
I20260806 06:45:28.165902 3170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:28.166174 3170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:28.167337 3170 hybrid_clock.cc:648] HybridClock initialized: now 1785998728167326 us; error 46 us; skew 500 ppm
Aug 06 06:45:28 dist-test-slave-1xrh krb5kdc[3157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:28.173781 3170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:28.174154 3170 webserver.cc:460] Webserver started at http://127.0.0.1:42271/ using document root <none> and password file <none>
I20260806 06:45:28.174387 3170 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:28.174432 3170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:28.174597 3170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:28.175697 3170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec"
format_stamp: "Formatted at 2026-08-06 06:45:28 on dist-test-slave-1xrh"
server_key: "22a4f41979f561836c2f0f8b38d7087b"
server_key_iv: "01ee81dfc50244847e412bd93815f5cb"
server_key_version: "encryptionkey@0"
I20260806 06:45:28.176043 3170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec"
format_stamp: "Formatted at 2026-08-06 06:45:28 on dist-test-slave-1xrh"
server_key: "22a4f41979f561836c2f0f8b38d7087b"
server_key_iv: "01ee81dfc50244847e412bd93815f5cb"
server_key_version: "encryptionkey@0"
I20260806 06:45:28.177333 3170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:28.178049 3185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:28.178220 3170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:28.178319 3170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec"
format_stamp: "Formatted at 2026-08-06 06:45:28 on dist-test-slave-1xrh"
server_key: "22a4f41979f561836c2f0f8b38d7087b"
server_key_iv: "01ee81dfc50244847e412bd93815f5cb"
server_key_version: "encryptionkey@0"
I20260806 06:45:28.178393 3170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:28.191852 3170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:28.194069 3170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:28.194221 3170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:28.198285 3170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43285
I20260806 06:45:28.198320 3240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43285 every 8 connection(s)
I20260806 06:45:28.198658 3170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:28.199467 3241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:28.201577 3241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec: Bootstrap starting.
I20260806 06:45:28.202090 3241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:28.202340 3241 log.cc:824] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:28.202482 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3170
I20260806 06:45:28.202600 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:28.202790 14541 external_mini_cluster.cc:1428] Setting key 088ede3353df4ba9460525a112fd2251
I20260806 06:45:28.203169 3241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec: No bootstrap required, opened a new log
I20260806 06:45:28.204943 3241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } }
I20260806 06:45:28.205118 3241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:28.205158 3241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ec6cb75b314007b3d6d1ac1b09c0ec, State: Initialized, Role: FOLLOWER
I20260806 06:45:28.205289 3241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } }
I20260806 06:45:28.205386 3241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:28.205423 3241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:28.205475 3241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:28.206046 3241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } }
I20260806 06:45:28.206166 3241 leader_election.cc:302] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ec6cb75b314007b3d6d1ac1b09c0ec; no voters:
I20260806 06:45:28.206365 3241 leader_election.cc:288] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:28.206387 3247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:28.206507 3247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [term 1 LEADER]: Becoming Leader. State: Replica: 59ec6cb75b314007b3d6d1ac1b09c0ec, State: Running, Role: LEADER
I20260806 06:45:28.206627 3241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:28.206646 3247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } }
I20260806 06:45:28.206997 3247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ec6cb75b314007b3d6d1ac1b09c0ec. Latest consensus state: current_term: 1 leader_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } } }
I20260806 06:45:28.207165 3247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:28.207592 3257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:28.208109 3257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:28.208163 3248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ec6cb75b314007b3d6d1ac1b09c0ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43285 } } }
I20260806 06:45:28.208302 3248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:28.210112 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:28.203313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42869 (local address 127.0.0.1:43285)
0806 06:45:28.203478 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:28.203486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:28.203533 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:45:28.204930 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:28.204937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:28.204981 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:28.205093 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:28.205356 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:28.206823 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:28.207964 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:28.208626 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:28.208766 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:28.209504 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:28.209508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:28.209508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:28.209678 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:28.209708 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:28.209718 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:28.209818 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:28.209899 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:45:28.209928 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260806 06:45:28.211086 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:28.215368 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:28.212376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50209 (local address 127.0.0.1:43285)
0806 06:45:28.212408 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:28.212409 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:28.212475 (+ 66us) server_negotiation.cc:407] Connection header received
0806 06:45:28.212551 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:28.212551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:28.212559 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:28.212601 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:28.212725 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:28.213161 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:28.214175 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:28.214355 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:28.214401 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:28.215073 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:28.215074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:28.215074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:28.215077 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:28.215089 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:28.215093 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:28.215208 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:28.215277 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:45:28.215291 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:45:28.215993 3257 catalog_manager.cc:1347] Generated new cluster ID: 46843a39303846ec9a7b225674d7b32a
I20260806 06:45:28.216049 3257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:28.234957 3257 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:28.235533 3257 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:28.238185 3257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59ec6cb75b314007b3d6d1ac1b09c0ec: Generated new TSK 0
I20260806 06:45:28.238369 3257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:28.240397 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3170
W20260806 06:45:28.245779 3155 connection.cc:535] client connection to 127.0.0.1:43285 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43285): Connection reset by peer (error 104)
2026-08-06T06:45:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:28 dist-test-slave-1xrh krb5kdc[3274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:28 dist-test-slave-1xrh krb5kdc[3274](info): set up 2 sockets
Aug 06 06:45:28 dist-test-slave-1xrh krb5kdc[3274](info): commencing operation
krb5kdc: starting...
W20260806 06:45:29.298499 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:29 dist-test-slave-1xrh krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:29.417415 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:29.496592 3287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:29.496773 3287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:29.496794 3287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:29.497838 3287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:29.497870 3287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:29.497884 3287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:29.497896 3287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:29.497908 3287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:29.499238 3287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:29.499410 3287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:29.499565 3287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:29.504153 3295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:29.504114 3293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:29.504118 3292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:29.504164 3287 server_base.cc:1034] running on GCE node
I20260806 06:45:29.504547 3287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:29.504861 3287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:29.506031 3287 hybrid_clock.cc:648] HybridClock initialized: now 1785998729506020 us; error 42 us; skew 500 ppm
Aug 06 06:45:29 dist-test-slave-1xrh krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:29.508443 3287 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:29.508805 3287 webserver.cc:460] Webserver started at http://127.0.0.1:34101/ using document root <none> and password file <none>
I20260806 06:45:29.509011 3287 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:29.509054 3287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:29.509187 3287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:29.510146 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "25446c3ac350406ebb0db2ef8c857040"
format_stamp: "Formatted at 2026-08-06 06:45:29 on dist-test-slave-1xrh"
server_key: "5d1965c728bd74623abef5f7dfbcba84"
server_key_iv: "442c47c58322ac38540374fd040a0214"
server_key_version: "encryptionkey@0"
I20260806 06:45:29.510483 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "25446c3ac350406ebb0db2ef8c857040"
format_stamp: "Formatted at 2026-08-06 06:45:29 on dist-test-slave-1xrh"
server_key: "5d1965c728bd74623abef5f7dfbcba84"
server_key_iv: "442c47c58322ac38540374fd040a0214"
server_key_version: "encryptionkey@0"
I20260806 06:45:29.511929 3287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:29.512609 3304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:29.512775 3287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:29.512879 3287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "25446c3ac350406ebb0db2ef8c857040"
format_stamp: "Formatted at 2026-08-06 06:45:29 on dist-test-slave-1xrh"
server_key: "5d1965c728bd74623abef5f7dfbcba84"
server_key_iv: "442c47c58322ac38540374fd040a0214"
server_key_version: "encryptionkey@0"
I20260806 06:45:29.512950 3287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:29.516145 3287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:29.517860 3287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:29.518020 3287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:29.521821 3287 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42557
I20260806 06:45:29.521853 3358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42557 every 8 connection(s)
I20260806 06:45:29.522184 3287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:29.523017 3359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:29.525118 3359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040: Bootstrap starting.
I20260806 06:45:29.525657 3359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:29.525934 3359 log.cc:824] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:29.526618 3359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040: No bootstrap required, opened a new log
I20260806 06:45:29.528211 3359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } }
I20260806 06:45:29.528357 3359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:29.528403 3359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25446c3ac350406ebb0db2ef8c857040, State: Initialized, Role: FOLLOWER
I20260806 06:45:29.528565 3359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } }
I20260806 06:45:29.528630 3359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:29.528668 3359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:29.528716 3359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:29.529214 3359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } }
I20260806 06:45:29.529325 3359 leader_election.cc:302] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25446c3ac350406ebb0db2ef8c857040; no voters:
I20260806 06:45:29.529503 3359 leader_election.cc:288] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:29.529531 3362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:29.529737 3362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [term 1 LEADER]: Becoming Leader. State: Replica: 25446c3ac350406ebb0db2ef8c857040, State: Running, Role: LEADER
I20260806 06:45:29.529740 3359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:29.529838 3362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } }
I20260806 06:45:29.530227 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25446c3ac350406ebb0db2ef8c857040. Latest consensus state: current_term: 1 leader_uuid: "25446c3ac350406ebb0db2ef8c857040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } } }
I20260806 06:45:29.530395 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:29.530311 3363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25446c3ac350406ebb0db2ef8c857040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25446c3ac350406ebb0db2ef8c857040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42557 } } }
I20260806 06:45:29.530639 3363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:29.530771 3372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:29.531276 3372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:29.531665 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3287
I20260806 06:45:29.531759 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:29.531874 14541 external_mini_cluster.cc:1428] Setting key 77334fed02975e481094dfddf59690ae
I20260806 06:45:29.533464 3372 catalog_manager.cc:1347] Generated new cluster ID: afb13bb0ca9247ce9ff85e8b51dcb094
I20260806 06:45:29.533526 3372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:29.534205 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:29.532283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54155 (local address 127.0.0.1:42557)
0806 06:45:29.532503 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:29.532517 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:29.532548 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:29.532654 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:29.532660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:29.532697 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:29.532832 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:29.533768 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:29.533772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:29.533772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:29.533776 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:29.533819 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:29.533834 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:29.533918 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:29.534011 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:45:29.534050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260806 06:45:29.535149 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:29.539318 3372 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:29.539464 3372 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:29.540133 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:29.536647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43619 (local address 127.0.0.1:42557)
0806 06:45:29.539191 (+ 2544us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:29.539193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:29.539202 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:29.539220 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:29.539221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:29.539225 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:29.539275 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:29.539832 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:29.539833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:29.539833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:29.539837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:29.539849 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:29.539853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:29.539893 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:29.540057 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:45:29.540069 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2526}
I20260806 06:45:29.554641 3372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25446c3ac350406ebb0db2ef8c857040: Generated new TSK 0
I20260806 06:45:29.554824 3372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:29.564137 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3287
2026-08-06T06:45:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:29 dist-test-slave-1xrh krb5kdc[3392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:29 dist-test-slave-1xrh krb5kdc[3392](info): set up 2 sockets
Aug 06 06:45:29 dist-test-slave-1xrh krb5kdc[3392](info): commencing operation
krb5kdc: starting...
W20260806 06:45:30.607060 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:30.721640 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42885
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:30.802278 3407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:30.802529 3407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:30.802570 3407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:30.803763 3407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:30.803845 3407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:30.803872 3407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:30.803903 3407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:30.803927 3407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:30.805383 3407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:30.805650 3407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:30.805855 3407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:30.810549 3415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:30.810644 3412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:30.810541 3413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:30.810880 3407 server_base.cc:1034] running on GCE node
I20260806 06:45:30.811115 3407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:30.811441 3407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:30.812614 3407 hybrid_clock.cc:648] HybridClock initialized: now 1785998730812600 us; error 29 us; skew 500 ppm
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998730, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:30.814934 3407 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:30.815433 3407 webserver.cc:460] Webserver started at http://127.0.0.1:33539/ using document root <none> and password file <none>
I20260806 06:45:30.815671 3407 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:30.815739 3407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:30.815871 3407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:30.817184 3407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "8ad94f770c0a43e5b963a61e31b49a51"
format_stamp: "Formatted at 2026-08-06 06:45:30 on dist-test-slave-1xrh"
server_key: "4d9797e3ef65f7eb6b4a7a0649a8573a"
server_key_iv: "4bc115b40e96d656955ebf3a54b6d782"
server_key_version: "encryptionkey@0"
I20260806 06:45:30.817600 3407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "8ad94f770c0a43e5b963a61e31b49a51"
format_stamp: "Formatted at 2026-08-06 06:45:30 on dist-test-slave-1xrh"
server_key: "4d9797e3ef65f7eb6b4a7a0649a8573a"
server_key_iv: "4bc115b40e96d656955ebf3a54b6d782"
server_key_version: "encryptionkey@0"
I20260806 06:45:30.818877 3407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:30.819667 3422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:30.819850 3407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:30.819945 3407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "8ad94f770c0a43e5b963a61e31b49a51"
format_stamp: "Formatted at 2026-08-06 06:45:30 on dist-test-slave-1xrh"
server_key: "4d9797e3ef65f7eb6b4a7a0649a8573a"
server_key_iv: "4bc115b40e96d656955ebf3a54b6d782"
server_key_version: "encryptionkey@0"
I20260806 06:45:30.820032 3407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:30.832084 3407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:30.834193 3407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:30.834371 3407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:30.838284 3407 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42885
I20260806 06:45:30.838321 3476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42885 every 8 connection(s)
I20260806 06:45:30.838660 3407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:30.839558 3477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:30.841677 3477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51: Bootstrap starting.
I20260806 06:45:30.842228 3477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:30.842520 3477 log.cc:824] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:30.843252 3477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51: No bootstrap required, opened a new log
I20260806 06:45:30.844805 3477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } }
I20260806 06:45:30.844933 3477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:30.844982 3477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad94f770c0a43e5b963a61e31b49a51, State: Initialized, Role: FOLLOWER
I20260806 06:45:30.845165 3477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } }
I20260806 06:45:30.845245 3477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:30.845295 3477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:30.845345 3477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:30.845695 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3407
I20260806 06:45:30.845824 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:30.845932 3477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } }
I20260806 06:45:30.846010 14541 external_mini_cluster.cc:1428] Setting key 67bdbdc9c54fddc14160502c63827d10
I20260806 06:45:30.846031 3477 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ad94f770c0a43e5b963a61e31b49a51; no voters:
I20260806 06:45:30.846201 3477 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:30.846227 3482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:30.846342 3482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [term 1 LEADER]: Becoming Leader. State: Replica: 8ad94f770c0a43e5b963a61e31b49a51, State: Running, Role: LEADER
I20260806 06:45:30.846450 3477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:30.847231 3482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } }
I20260806 06:45:30.847836 3485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ad94f770c0a43e5b963a61e31b49a51. Latest consensus state: current_term: 1 leader_uuid: "8ad94f770c0a43e5b963a61e31b49a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } } }
I20260806 06:45:30.847987 3485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:30.848284 3484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ad94f770c0a43e5b963a61e31b49a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad94f770c0a43e5b963a61e31b49a51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42885 } } }
I20260806 06:45:30.848356 3501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:30.848388 3484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:30.848829 3501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:30.850368 3501 catalog_manager.cc:1347] Generated new cluster ID: e29e1b0c8f20468eaa891a11ec8e8acc
I20260806 06:45:30.850420 3501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:45:30.857877 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:30.846468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40511 (local address 127.0.0.1:42885)
0806 06:45:30.846635 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:30.846643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:30.846676 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:45:30.846803 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:30.846809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:30.846862 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:30.846978 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:30.847925 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:30.849225 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:30.850239 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:30.850813 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:30.850902 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:30.853631 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:30.853635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:30.853635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:30.853647 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:30.856159 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:30.856534 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:30.856537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:30.856537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:30.856642 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:30.856826 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:30.856827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:30.856828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:30.857118 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:30.857264 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:30.857505 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 06:45:30.857640 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260806 06:45:30.858438 3501 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:30.858951 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:30.859092 3501 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:30.867554 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:30.860492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59165 (local address 127.0.0.1:42885)
0806 06:45:30.860525 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:30.860527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:30.860645 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:45:30.860697 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:30.860697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:30.860705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:30.860759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:30.861357 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:30.862078 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:30.863158 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:30.863455 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:30.863513 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:30.864876 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:30.864877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:30.864877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:30.864883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:30.866389 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:30.866780 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:30.866781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:30.866781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:30.866834 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:30.867042 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:30.867043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:30.867043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:30.867180 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:30.867271 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:30.867398 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 06:45:30.867470 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:30.868875 3501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ad94f770c0a43e5b963a61e31b49a51: Generated new TSK 0
I20260806 06:45:30.869063 3501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:30.873939 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3407
2026-08-06T06:45:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3511](info): set up 2 sockets
Aug 06 06:45:30 dist-test-slave-1xrh krb5kdc[3511](info): commencing operation
krb5kdc: starting...
W20260806 06:45:31.909957 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:31 dist-test-slave-1xrh krb5kdc[3511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:32.022104 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:32.100687 3524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:32.100867 3524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:32.100888 3524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:32.102059 3524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:32.102100 3524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:32.102113 3524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:32.102126 3524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:32.102138 3524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:32.103550 3524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:32.103725 3524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:32.103892 3524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:32.108593 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:32.108601 3529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:32.108608 3532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:32.108795 3524 server_base.cc:1034] running on GCE node
I20260806 06:45:32.109048 3524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:32.109303 3524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:32.110466 3524 hybrid_clock.cc:648] HybridClock initialized: now 1785998732110455 us; error 47 us; skew 500 ppm
Aug 06 06:45:32 dist-test-slave-1xrh krb5kdc[3511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998732, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:32.112826 3524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:32.113282 3524 webserver.cc:460] Webserver started at http://127.0.0.1:42155/ using document root <none> and password file <none>
I20260806 06:45:32.113504 3524 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:32.113577 3524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:32.113721 3524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:32.114711 3524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "f7fb20f3444840d7bfa9e2edf11bc053"
format_stamp: "Formatted at 2026-08-06 06:45:32 on dist-test-slave-1xrh"
server_key: "3c5e27c19ebf9b0e4ddd1af362d1bd7f"
server_key_iv: "cf023e259ed21dae1a88898d5ae89f86"
server_key_version: "encryptionkey@0"
I20260806 06:45:32.115054 3524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "f7fb20f3444840d7bfa9e2edf11bc053"
format_stamp: "Formatted at 2026-08-06 06:45:32 on dist-test-slave-1xrh"
server_key: "3c5e27c19ebf9b0e4ddd1af362d1bd7f"
server_key_iv: "cf023e259ed21dae1a88898d5ae89f86"
server_key_version: "encryptionkey@0"
I20260806 06:45:32.116407 3524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:32.117151 3539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:32.117336 3524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:32.117429 3524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "f7fb20f3444840d7bfa9e2edf11bc053"
format_stamp: "Formatted at 2026-08-06 06:45:32 on dist-test-slave-1xrh"
server_key: "3c5e27c19ebf9b0e4ddd1af362d1bd7f"
server_key_iv: "cf023e259ed21dae1a88898d5ae89f86"
server_key_version: "encryptionkey@0"
I20260806 06:45:32.117517 3524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:32.129289 3524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:32.130882 3524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:32.131034 3524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:32.134888 3524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44585
I20260806 06:45:32.134919 3595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44585 every 8 connection(s)
I20260806 06:45:32.135325 3524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:32.136126 3596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:32.136262 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3524
I20260806 06:45:32.136389 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:32.136570 14541 external_mini_cluster.cc:1428] Setting key 16740debb495b12467f730d948fb9755
I20260806 06:45:32.139129 3596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053: Bootstrap starting.
I20260806 06:45:32.139820 3596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:32.140128 3596 log.cc:824] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:32.140878 3596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053: No bootstrap required, opened a new log
I20260806 06:45:32.141825 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:32.137065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44677 (local address 127.0.0.1:44585)
0806 06:45:32.137243 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:32.137249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:32.137272 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:45:32.137676 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:32.137682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:32.137724 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:32.137825 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:32.138070 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:32.138878 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:32.139980 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:32.140391 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:32.140528 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:32.141251 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:32.141254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:32.141255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:32.141406 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:32.141432 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:32.141441 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:32.141541 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:32.141613 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:45:32.141671 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260806 06:45:32.142835 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:32.143054 3596 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260806 06:45:32.143250 3596 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:32.143301 3596 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7fb20f3444840d7bfa9e2edf11bc053, State: Initialized, Role: FOLLOWER
I20260806 06:45:32.143419 3596 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260806 06:45:32.143504 3596 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:32.143545 3596 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:32.143594 3596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:32.144194 3596 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260806 06:45:32.144311 3596 leader_election.cc:302] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7fb20f3444840d7bfa9e2edf11bc053; no voters:
I20260806 06:45:32.144551 3596 leader_election.cc:288] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:32.144579 3606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:32.144748 3606 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [term 1 LEADER]: Becoming Leader. State: Replica: f7fb20f3444840d7bfa9e2edf11bc053, State: Running, Role: LEADER
I20260806 06:45:32.144843 3596 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:32.144877 3606 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260806 06:45:32.145273 3606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7fb20f3444840d7bfa9e2edf11bc053. Latest consensus state: current_term: 1 leader_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260806 06:45:32.145519 3606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:32.145443 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fb20f3444840d7bfa9e2edf11bc053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260806 06:45:32.145937 3615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:32.145944 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:32.146529 3615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:32.147626 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:32.144364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56649 (local address 127.0.0.1:44585)
0806 06:45:32.144402 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:32.144404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:32.144452 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:45:32.144528 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:32.144528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:32.144537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:32.144581 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:32.144694 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:32.145343 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:32.146314 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:32.146623 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:32.146694 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:32.147362 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:32.147363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:32.147364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:32.147369 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:32.147383 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:32.147387 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:32.147449 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:32.147537 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 06:45:32.147559 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:45:32.148156 3615 catalog_manager.cc:1347] Generated new cluster ID: 3a2871bc330949ccac242feab82156c5
I20260806 06:45:32.148205 3615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:32.171073 3615 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:32.171842 3615 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:32.174177 3615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7fb20f3444840d7bfa9e2edf11bc053: Generated new TSK 0
I20260806 06:45:32.174352 3615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:32.181762 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3524
2026-08-06T06:45:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:32 dist-test-slave-1xrh krb5kdc[3629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:32 dist-test-slave-1xrh krb5kdc[3629](info): set up 2 sockets
Aug 06 06:45:32 dist-test-slave-1xrh krb5kdc[3629](info): commencing operation
krb5kdc: starting...
W20260806 06:45:33.294377 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:33 dist-test-slave-1xrh krb5kdc[3629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:33.413935 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:33.492215 3642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:33.492395 3642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:33.492417 3642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:33.493556 3642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:33.493602 3642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:33.493616 3642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:33.493670 3642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:33.493690 3642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:33.495044 3642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:33.495316 3642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:33.495534 3642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:45:33.500335 3642 server_base.cc:1034] running on GCE node
W20260806 06:45:33.500304 3648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:33.500393 3647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:33.500298 3650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:33.500746 3642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:33.501003 3642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:33.502144 3642 hybrid_clock.cc:648] HybridClock initialized: now 1785998733502129 us; error 31 us; skew 500 ppm
Aug 06 06:45:33 dist-test-slave-1xrh krb5kdc[3629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:33.509406 3642 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:33.509807 3642 webserver.cc:460] Webserver started at http://127.0.0.1:34527/ using document root <none> and password file <none>
I20260806 06:45:33.510033 3642 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:33.510100 3642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:33.510246 3642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:33.511266 3642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "7181d4dcefbc48f8a7a78f78f56a5a50"
format_stamp: "Formatted at 2026-08-06 06:45:33 on dist-test-slave-1xrh"
server_key: "c318d33c87e6fde1706bbf779e6cc41a"
server_key_iv: "e4ee84ba7bf5e107c012c1500bddfeb4"
server_key_version: "encryptionkey@0"
I20260806 06:45:33.511600 3642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "7181d4dcefbc48f8a7a78f78f56a5a50"
format_stamp: "Formatted at 2026-08-06 06:45:33 on dist-test-slave-1xrh"
server_key: "c318d33c87e6fde1706bbf779e6cc41a"
server_key_iv: "e4ee84ba7bf5e107c012c1500bddfeb4"
server_key_version: "encryptionkey@0"
I20260806 06:45:33.512856 3642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:45:33.513535 3657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:33.513702 3642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:33.513800 3642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "7181d4dcefbc48f8a7a78f78f56a5a50"
format_stamp: "Formatted at 2026-08-06 06:45:33 on dist-test-slave-1xrh"
server_key: "c318d33c87e6fde1706bbf779e6cc41a"
server_key_iv: "e4ee84ba7bf5e107c012c1500bddfeb4"
server_key_version: "encryptionkey@0"
I20260806 06:45:33.513875 3642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:33.517093 3642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:33.518751 3642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:33.518896 3642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:33.522603 3642 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43573
I20260806 06:45:33.522644 3713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43573 every 8 connection(s)
I20260806 06:45:33.522969 3642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:33.523859 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:33.525930 3714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: Bootstrap starting.
I20260806 06:45:33.526479 3714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:33.526760 3714 log.cc:824] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:33.527458 3714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: No bootstrap required, opened a new log
I20260806 06:45:33.527704 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3642
I20260806 06:45:33.527830 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:33.528021 14541 external_mini_cluster.cc:1428] Setting key e932f916adccd7cb5a41955db446ee30
I20260806 06:45:33.529707 3714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } }
I20260806 06:45:33.529899 3714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:33.529932 3714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7181d4dcefbc48f8a7a78f78f56a5a50, State: Initialized, Role: FOLLOWER
I20260806 06:45:33.530058 3714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } }
I20260806 06:45:33.530138 3714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:33.530223 3714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:33.530284 3714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:33.530908 3714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } }
I20260806 06:45:33.531054 3714 leader_election.cc:302] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7181d4dcefbc48f8a7a78f78f56a5a50; no voters:
I20260806 06:45:33.531211 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:33.528489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36813 (local address 127.0.0.1:43573)
0806 06:45:33.528703 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:33.528712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:33.528750 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:45:33.529894 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:33.529902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:33.529936 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:33.530078 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:33.530718 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:33.530722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:33.530722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:33.530727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:33.530788 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:33.530802 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:33.530880 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:33.530932 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:45:33.530971 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260806 06:45:33.531245 3714 leader_election.cc:288] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:33.531308 3719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:33.531461 3719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [term 1 LEADER]: Becoming Leader. State: Replica: 7181d4dcefbc48f8a7a78f78f56a5a50, State: Running, Role: LEADER
I20260806 06:45:33.531469 3714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:33.531603 3719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } }
I20260806 06:45:33.531999 3719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7181d4dcefbc48f8a7a78f78f56a5a50. Latest consensus state: current_term: 1 leader_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } } }
I20260806 06:45:33.532060 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:33.532227 3719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:33.532166 3720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7181d4dcefbc48f8a7a78f78f56a5a50" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43573 } } }
I20260806 06:45:33.532440 3720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50 [sys.catalog]: This master's current role is: LEADER
W20260806 06:45:33.533248 3735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:33.533308 3735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:33.533380 3728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:33.533555 3728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:33.534941 3728 catalog_manager.cc:1347] Generated new cluster ID: 52c782d3dcab4528a99c445eccd8c225
I20260806 06:45:33.535014 3728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:33.535563 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:33.534260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45269 (local address 127.0.0.1:43573)
0806 06:45:33.534295 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:33.534296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:33.534603 (+ 307us) server_negotiation.cc:407] Connection header received
0806 06:45:33.534670 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:33.534670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:33.534674 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:33.534717 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:33.535364 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:33.535365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:33.535365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:33.535369 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:33.535380 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:33.535384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:33.535432 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:33.535484 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:45:33.535498 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:45:33.555059 3728 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:33.555235 3728 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:33.559064 3728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7181d4dcefbc48f8a7a78f78f56a5a50: Generated new TSK 0
I20260806 06:45:33.559255 3728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:33.561340 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3642
2026-08-06T06:45:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:33 dist-test-slave-1xrh krb5kdc[3745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:33 dist-test-slave-1xrh krb5kdc[3745](info): set up 2 sockets
Aug 06 06:45:33 dist-test-slave-1xrh krb5kdc[3745](info): commencing operation
krb5kdc: starting...
W20260806 06:45:34.606496 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:34.721213 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:34.804965 3758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:34.805174 3758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:34.805198 3758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:34.806479 3758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:34.806524 3758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:34.806541 3758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:34.806597 3758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:34.806622 3758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:34.808487 3758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:34.808759 3758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:34.809023 3758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:34.814204 3767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:34.814213 3769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:34.814260 3758 server_base.cc:1034] running on GCE node
W20260806 06:45:34.814222 3765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:34.814618 3758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:34.814939 3758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:34.816085 3758 hybrid_clock.cc:648] HybridClock initialized: now 1785998734816074 us; error 33 us; skew 500 ppm
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:34.818612 3758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:34.819235 3758 webserver.cc:460] Webserver started at http://127.0.0.1:35603/ using document root <none> and password file <none>
I20260806 06:45:34.819489 3758 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:34.819572 3758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:34.819766 3758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:34.820871 3758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "33bf080fa01b476a8d665b39d31e96a6"
format_stamp: "Formatted at 2026-08-06 06:45:34 on dist-test-slave-1xrh"
server_key: "1572a216c442b17e6f2ba5ff4e2be3d8"
server_key_iv: "e911beacca72d903c427e6ee62815321"
server_key_version: "encryptionkey@0"
I20260806 06:45:34.821249 3758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "33bf080fa01b476a8d665b39d31e96a6"
format_stamp: "Formatted at 2026-08-06 06:45:34 on dist-test-slave-1xrh"
server_key: "1572a216c442b17e6f2ba5ff4e2be3d8"
server_key_iv: "e911beacca72d903c427e6ee62815321"
server_key_version: "encryptionkey@0"
I20260806 06:45:34.822631 3758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:34.823531 3776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:34.823710 3758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:45:34.823835 3758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "33bf080fa01b476a8d665b39d31e96a6"
format_stamp: "Formatted at 2026-08-06 06:45:34 on dist-test-slave-1xrh"
server_key: "1572a216c442b17e6f2ba5ff4e2be3d8"
server_key_iv: "e911beacca72d903c427e6ee62815321"
server_key_version: "encryptionkey@0"
I20260806 06:45:34.823915 3758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:34.842238 3758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:34.844107 3758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:34.844265 3758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:34.848356 3758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46099
I20260806 06:45:34.848385 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46099 every 8 connection(s)
I20260806 06:45:34.848716 3758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:34.849498 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:34.851651 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6: Bootstrap starting.
I20260806 06:45:34.852174 3830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:34.852397 3830 log.cc:824] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:34.853030 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6: No bootstrap required, opened a new log
I20260806 06:45:34.854519 3830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260806 06:45:34.854645 3830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:34.854666 3830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bf080fa01b476a8d665b39d31e96a6, State: Initialized, Role: FOLLOWER
I20260806 06:45:34.854760 3830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260806 06:45:34.854841 3830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:34.854871 3830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:34.854920 3830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:34.855373 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3758
I20260806 06:45:34.855518 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:34.855566 3830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260806 06:45:34.855705 3830 leader_election.cc:302] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33bf080fa01b476a8d665b39d31e96a6; no voters:
I20260806 06:45:34.855733 14541 external_mini_cluster.cc:1428] Setting key 3f58883cee689b5445018fd56401c9f2
I20260806 06:45:34.855876 3830 leader_election.cc:288] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:34.855916 3834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:34.856119 3834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [term 1 LEADER]: Becoming Leader. State: Replica: 33bf080fa01b476a8d665b39d31e96a6, State: Running, Role: LEADER
I20260806 06:45:34.856148 3830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:34.856272 3834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260806 06:45:34.857023 3835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33bf080fa01b476a8d665b39d31e96a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } } }
I20260806 06:45:34.857143 3835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:34.857299 3836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33bf080fa01b476a8d665b39d31e96a6. Latest consensus state: current_term: 1 leader_uuid: "33bf080fa01b476a8d665b39d31e96a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bf080fa01b476a8d665b39d31e96a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } } }
I20260806 06:45:34.857404 3836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:34.857338 3845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:34.857952 3845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:45:34.863057 3845 catalog_manager.cc:1347] Generated new cluster ID: a04f52ecbd59447287c1811d32e06e40
I20260806 06:45:34.863116 3845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:34.866634 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:34.856209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:34627 (local address 127.0.0.1:46099)
0806 06:45:34.856629 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:34.856638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:34.856669 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:34.856783 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:34.856797 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:34.856887 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:34.857041 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:34.858138 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:34.859382 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:34.860150 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:34.860580 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:34.860746 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:34.863391 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:34.863395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:34.863395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:34.863406 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:34.865067 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:34.865467 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:34.865470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:34.865470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:34.865576 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:34.865817 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:34.865818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:34.865818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:34.866045 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:34.866234 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:34.866350 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:45:34.866457 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":69,"threads_started":1}
I20260806 06:45:34.867515 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:34.874744 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:34.868866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:33039 (local address 127.0.0.1:46099)
0806 06:45:34.868903 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:34.868904 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:34.869034 (+ 130us) server_negotiation.cc:407] Connection header received
0806 06:45:34.869104 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:34.869105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:34.869115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:34.869162 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:34.869647 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:34.870104 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:34.870760 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:34.870974 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:34.871050 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:34.872182 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:34.872183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:34.872184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:34.872190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:34.873571 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:34.873927 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:34.873928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:34.873929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:34.873983 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:34.874238 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:34.874240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:34.874240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:34.874346 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:34.874465 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:34.874573 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:45:34.874635 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:45:34.888221 3845 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:34.888728 3845 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:34.896907 3845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33bf080fa01b476a8d665b39d31e96a6: Generated new TSK 0
I20260806 06:45:34.897082 3845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:34.900265 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3758
W20260806 06:45:34.905429 3743 connection.cc:535] client connection to 127.0.0.1:46099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46099): Connection reset by peer (error 104)
2026-08-06T06:45:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3863](info): set up 2 sockets
Aug 06 06:45:34 dist-test-slave-1xrh krb5kdc[3863](info): commencing operation
krb5kdc: starting...
W20260806 06:45:35.955451 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:36 dist-test-slave-1xrh krb5kdc[3863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:36.069653 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:36.147981 3876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:36.148233 3876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:36.148275 3876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:36.149537 3876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:36.149614 3876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:36.149641 3876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:36.149667 3876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:36.149699 3876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:36.151221 3876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:36.151530 3876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:36.151757 3876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:36.156473 3885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:36.156508 3882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:36.156481 3881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:36.156594 3876 server_base.cc:1034] running on GCE node
I20260806 06:45:36.156916 3876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:36.157192 3876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:36.158366 3876 hybrid_clock.cc:648] HybridClock initialized: now 1785998736158350 us; error 36 us; skew 500 ppm
Aug 06 06:45:36 dist-test-slave-1xrh krb5kdc[3863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998736, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:36.160840 3876 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:36.161254 3876 webserver.cc:460] Webserver started at http://127.0.0.1:33009/ using document root <none> and password file <none>
I20260806 06:45:36.161475 3876 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:36.161556 3876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:36.161705 3876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:36.162657 3876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d"
format_stamp: "Formatted at 2026-08-06 06:45:36 on dist-test-slave-1xrh"
server_key: "6c03863dd3dd3615e305ac0cadc54d6d"
server_key_iv: "cb716a1f7125cd99494333602914d401"
server_key_version: "encryptionkey@0"
I20260806 06:45:36.163028 3876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d"
format_stamp: "Formatted at 2026-08-06 06:45:36 on dist-test-slave-1xrh"
server_key: "6c03863dd3dd3615e305ac0cadc54d6d"
server_key_iv: "cb716a1f7125cd99494333602914d401"
server_key_version: "encryptionkey@0"
I20260806 06:45:36.164340 3876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:36.165038 3892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:36.165241 3876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:36.165337 3876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d"
format_stamp: "Formatted at 2026-08-06 06:45:36 on dist-test-slave-1xrh"
server_key: "6c03863dd3dd3615e305ac0cadc54d6d"
server_key_iv: "cb716a1f7125cd99494333602914d401"
server_key_version: "encryptionkey@0"
I20260806 06:45:36.165411 3876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:36.198115 3876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:36.200637 3876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:36.200812 3876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:36.204870 3876 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34293
I20260806 06:45:36.204907 3947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34293 every 8 connection(s)
I20260806 06:45:36.205264 3876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:36.206135 3948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:36.208346 3948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d: Bootstrap starting.
I20260806 06:45:36.208905 3948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:36.209132 3948 log.cc:824] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:36.209872 3948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d: No bootstrap required, opened a new log
I20260806 06:45:36.211549 3948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260806 06:45:36.211674 3948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:36.211695 3948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fc45dafdbe41d28ac4f2a4da8dc39d, State: Initialized, Role: FOLLOWER
I20260806 06:45:36.211822 3948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260806 06:45:36.211901 3948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:36.211922 3948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:36.211949 3948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:36.212389 3948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260806 06:45:36.212467 3948 leader_election.cc:302] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85fc45dafdbe41d28ac4f2a4da8dc39d; no voters:
I20260806 06:45:36.212587 3948 leader_election.cc:288] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:36.212651 3951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:36.212841 3951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [term 1 LEADER]: Becoming Leader. State: Replica: 85fc45dafdbe41d28ac4f2a4da8dc39d, State: Running, Role: LEADER
I20260806 06:45:36.212884 3948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:36.212935 3951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260806 06:45:36.213292 3953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85fc45dafdbe41d28ac4f2a4da8dc39d. Latest consensus state: current_term: 1 leader_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260806 06:45:36.213373 3953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:36.213415 3952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fc45dafdbe41d28ac4f2a4da8dc39d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260806 06:45:36.213475 3952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:36.213645 3960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:36.213980 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3876
I20260806 06:45:36.214116 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:36.214318 14541 external_mini_cluster.cc:1428] Setting key 4629ac17f9f71c3fc92f862687ef6747
I20260806 06:45:36.214699 3960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:36.216228 3960 catalog_manager.cc:1347] Generated new cluster ID: 99e7ee0281284e3c9e72cf009337a56d
I20260806 06:45:36.216279 3960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:36.220362 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:36.214960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:51891 (local address 127.0.0.1:34293)
0806 06:45:36.215272 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:36.215280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:36.215352 (+ 72us) server_negotiation.cc:407] Connection header received
0806 06:45:36.215451 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:36.215456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:36.215497 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:36.215637 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:36.215824 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:36.217108 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:36.218160 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:36.218788 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:36.218952 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:36.219621 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:36.219624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:36.219625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:36.219844 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:36.219880 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:36.219904 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:36.220035 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:36.220119 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:45:36.220144 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":77,"threads_started":1}
I20260806 06:45:36.221240 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:36.226456 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:36.222793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:56549 (local address 127.0.0.1:34293)
0806 06:45:36.222823 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:36.222824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:36.222969 (+ 145us) server_negotiation.cc:407] Connection header received
0806 06:45:36.223061 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:36.223062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:36.223072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:36.223101 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:36.223345 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:36.224075 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:36.225103 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:36.225404 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:36.225446 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:36.226188 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:36.226190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:36.226190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:36.226194 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:36.226218 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:36.226222 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:36.226252 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:36.226376 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:45:36.226388 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:45:36.235476 3960 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:36.236106 3960 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:36.239491 3960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85fc45dafdbe41d28ac4f2a4da8dc39d: Generated new TSK 0
I20260806 06:45:36.239672 3960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:36.244050 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3876
W20260806 06:45:36.249840 3861 connection.cc:535] client connection to 127.0.0.1:34293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34293): Connection reset by peer (error 104)
2026-08-06T06:45:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:36 dist-test-slave-1xrh krb5kdc[3980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:36 dist-test-slave-1xrh krb5kdc[3980](info): set up 2 sockets
Aug 06 06:45:36 dist-test-slave-1xrh krb5kdc[3980](info): commencing operation
krb5kdc: starting...
W20260806 06:45:37.332062 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:37 dist-test-slave-1xrh krb5kdc[3980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:37.451787 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:37.530673 3994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:37.530906 3994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:37.530946 3994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:37.532186 3994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:37.532255 3994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:37.532279 3994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:37.532299 3994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:37.532328 3994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:37.533782 3994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:37.534068 3994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:37.534294 3994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:37.538959 4002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:37.538985 3999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:37.538959 4000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:37.539230 3994 server_base.cc:1034] running on GCE node
I20260806 06:45:37.539461 3994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:37.539736 3994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:37.540894 3994 hybrid_clock.cc:648] HybridClock initialized: now 1785998737540879 us; error 28 us; skew 500 ppm
Aug 06 06:45:37 dist-test-slave-1xrh krb5kdc[3980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998737, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:37.543279 3994 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:37.543731 3994 webserver.cc:460] Webserver started at http://127.0.0.1:33189/ using document root <none> and password file <none>
I20260806 06:45:37.543991 3994 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:37.544061 3994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:37.544195 3994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:37.545176 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568"
format_stamp: "Formatted at 2026-08-06 06:45:37 on dist-test-slave-1xrh"
server_key: "6d98b4817ffa1a1a98cb242dbf99e560"
server_key_iv: "b8b3d002c650ff56f514547bb0990bd6"
server_key_version: "encryptionkey@0"
I20260806 06:45:37.545534 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568"
format_stamp: "Formatted at 2026-08-06 06:45:37 on dist-test-slave-1xrh"
server_key: "6d98b4817ffa1a1a98cb242dbf99e560"
server_key_iv: "b8b3d002c650ff56f514547bb0990bd6"
server_key_version: "encryptionkey@0"
I20260806 06:45:37.546801 3994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:37.547576 4009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:37.547739 3994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:37.547837 3994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568"
format_stamp: "Formatted at 2026-08-06 06:45:37 on dist-test-slave-1xrh"
server_key: "6d98b4817ffa1a1a98cb242dbf99e560"
server_key_iv: "b8b3d002c650ff56f514547bb0990bd6"
server_key_version: "encryptionkey@0"
I20260806 06:45:37.547915 3994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:37.551007 3994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:37.552716 3994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:37.552867 3994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:37.556825 3994 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32949
I20260806 06:45:37.556854 4064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32949 every 8 connection(s)
I20260806 06:45:37.557166 3994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:37.557935 4065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:37.560050 4065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568: Bootstrap starting.
I20260806 06:45:37.560590 4065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:37.560876 4065 log.cc:824] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:37.561520 4065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568: No bootstrap required, opened a new log
I20260806 06:45:37.563169 4065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260806 06:45:37.563299 4065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:37.563323 4065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6e64cc30cf4e7a82b0e0f790bc6568, State: Initialized, Role: FOLLOWER
I20260806 06:45:37.563479 4065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260806 06:45:37.563544 4065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:37.563566 4065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:37.563632 4065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:37.564154 4065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260806 06:45:37.564268 4065 leader_election.cc:302] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d6e64cc30cf4e7a82b0e0f790bc6568; no voters:
I20260806 06:45:37.564431 4065 leader_election.cc:288] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:37.564473 4068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:37.564654 4068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [term 1 LEADER]: Becoming Leader. State: Replica: 4d6e64cc30cf4e7a82b0e0f790bc6568, State: Running, Role: LEADER
I20260806 06:45:37.564711 4065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:37.564803 4068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260806 06:45:37.565546 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 3994
I20260806 06:45:37.565768 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:37.566144 14541 external_mini_cluster.cc:1428] Setting key 47b29eab55d03030b2e10e0795b3cf4a
I20260806 06:45:37.566357 4069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } } }
I20260806 06:45:37.566368 4070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d6e64cc30cf4e7a82b0e0f790bc6568. Latest consensus state: current_term: 1 leader_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6e64cc30cf4e7a82b0e0f790bc6568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } } }
I20260806 06:45:37.566468 4069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:37.566515 4070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:37.566818 4078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:37.567642 4078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:37.568774 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:37.566781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:34651 (local address 127.0.0.1:32949)
0806 06:45:37.567193 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:37.567202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:37.567237 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:45:37.567353 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:37.567359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:37.567381 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:37.567516 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:37.568341 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:37.568345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:37.568345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:37.568349 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:37.568394 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:37.568405 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:37.568475 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:37.568545 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:45:37.568591 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":131,"threads_started":1}
I20260806 06:45:37.569250 4078 catalog_manager.cc:1347] Generated new cluster ID: 9c8a1af0d667468c8ea1693750beeac5
I20260806 06:45:37.569303 4078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:37.569762 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:37.572193 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:37.571180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:40703 (local address 127.0.0.1:32949)
0806 06:45:37.571210 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:37.571211 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:37.571355 (+ 144us) server_negotiation.cc:407] Connection header received
0806 06:45:37.571413 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:37.571414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:37.571418 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:37.571471 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:37.572009 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:37.572010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:37.572011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:37.572014 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:37.572027 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:37.572031 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:37.572084 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:37.572125 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:45:37.572137 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:45:37.573714 4078 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:37.573860 4078 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:37.577368 4078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d6e64cc30cf4e7a82b0e0f790bc6568: Generated new TSK 0
I20260806 06:45:37.577538 4078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:37.581753 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 3994
2026-08-06T06:45:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:37 dist-test-slave-1xrh krb5kdc[4096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:37 dist-test-slave-1xrh krb5kdc[4096](info): set up 2 sockets
Aug 06 06:45:37 dist-test-slave-1xrh krb5kdc[4096](info): commencing operation
krb5kdc: starting...
W20260806 06:45:38.630652 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:38 dist-test-slave-1xrh krb5kdc[4096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:38.743028 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:38.823072 4109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:38.823336 4109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:38.823359 4109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:38.824518 4109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:38.824569 4109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:38.824584 4109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:38.824633 4109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:38.824657 4109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:38.826078 4109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:38.826310 4109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:38.826530 4109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:38.831096 4115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:38.831153 4116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:38.831264 4109 server_base.cc:1034] running on GCE node
W20260806 06:45:38.831193 4118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:38.831573 4109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:38.831856 4109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:38.833039 4109 hybrid_clock.cc:648] HybridClock initialized: now 1785998738833009 us; error 35 us; skew 500 ppm
Aug 06 06:45:38 dist-test-slave-1xrh krb5kdc[4096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998738, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:38.840024 4109 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:38.840451 4109 webserver.cc:460] Webserver started at http://127.0.0.1:40401/ using document root <none> and password file <none>
I20260806 06:45:38.840662 4109 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:38.840729 4109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:38.840874 4109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:38.841917 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "592fb32b05dc48ae8538f1e71ed5a995"
format_stamp: "Formatted at 2026-08-06 06:45:38 on dist-test-slave-1xrh"
server_key: "cd9adadecdede0fcd710182689588409"
server_key_iv: "e6bc7e7188351ddb4e2cabbb9a0cb59a"
server_key_version: "encryptionkey@0"
I20260806 06:45:38.842281 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "592fb32b05dc48ae8538f1e71ed5a995"
format_stamp: "Formatted at 2026-08-06 06:45:38 on dist-test-slave-1xrh"
server_key: "cd9adadecdede0fcd710182689588409"
server_key_iv: "e6bc7e7188351ddb4e2cabbb9a0cb59a"
server_key_version: "encryptionkey@0"
I20260806 06:45:38.843686 4109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:45:38.844408 4125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:38.844583 4109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:38.844682 4109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "592fb32b05dc48ae8538f1e71ed5a995"
format_stamp: "Formatted at 2026-08-06 06:45:38 on dist-test-slave-1xrh"
server_key: "cd9adadecdede0fcd710182689588409"
server_key_iv: "e6bc7e7188351ddb4e2cabbb9a0cb59a"
server_key_version: "encryptionkey@0"
I20260806 06:45:38.844763 4109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:38.866422 4109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:38.868865 4109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:38.869057 4109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:38.872997 4109 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43867
I20260806 06:45:38.873035 4180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43867 every 8 connection(s)
I20260806 06:45:38.873418 4109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:38.874284 4181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:38.876533 4181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: Bootstrap starting.
I20260806 06:45:38.877135 4181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:38.877242 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4109
I20260806 06:45:38.877354 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:38.877462 4181 log.cc:824] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:38.877570 14541 external_mini_cluster.cc:1428] Setting key e7b0f0f4e7c7cad6fd3a320ca372ae23
I20260806 06:45:38.878252 4181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: No bootstrap required, opened a new log
I20260806 06:45:38.880043 4181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260806 06:45:38.880199 4181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:38.880247 4181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592fb32b05dc48ae8538f1e71ed5a995, State: Initialized, Role: FOLLOWER
I20260806 06:45:38.880386 4181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260806 06:45:38.880465 4181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:38.880506 4181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:38.880565 4181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:38.881080 4181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260806 06:45:38.881160 4181 leader_election.cc:302] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 592fb32b05dc48ae8538f1e71ed5a995; no voters:
I20260806 06:45:38.881327 4181 leader_election.cc:288] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:38.881347 4187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:38.881451 4187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [term 1 LEADER]: Becoming Leader. State: Replica: 592fb32b05dc48ae8538f1e71ed5a995, State: Running, Role: LEADER
I20260806 06:45:38.881567 4181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:38.881589 4187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } }
I20260806 06:45:38.881901 4187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 592fb32b05dc48ae8538f1e71ed5a995. Latest consensus state: current_term: 1 leader_uuid: "592fb32b05dc48ae8538f1e71ed5a995" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } } }
I20260806 06:45:38.881979 4187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:38.881914 4188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "592fb32b05dc48ae8538f1e71ed5a995" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "592fb32b05dc48ae8538f1e71ed5a995" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43867 } } }
I20260806 06:45:38.882032 4188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995 [sys.catalog]: This master's current role is: LEADER
W20260806 06:45:38.883303 4202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:38.883558 4202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:38.883605 4192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:38.883752 4192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:38.885270 4192 catalog_manager.cc:1347] Generated new cluster ID: a1c7168477984ab18eea1cb8b0817c66
I20260806 06:45:38.885321 4192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:45:38 dist-test-slave-1xrh krb5kdc[4096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:45:38.889518 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:38.878191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:39101 (local address 127.0.0.1:43867)
0806 06:45:38.878369 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:38.878379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:38.878434 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:45:38.879405 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:38.879413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:38.879462 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:38.879610 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:38.880236 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:38.881680 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:38.882761 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:38.883417 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:38.883547 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:38.886300 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:38.886304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:38.886304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:38.886314 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:38.887957 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:38.888344 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:38.888346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:38.888347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:38.888433 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:38.888693 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:38.888694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:38.888694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:38.888913 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:38.889082 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:38.889229 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:45:38.889336 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260806 06:45:38.890394 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:38.897785 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:38.891748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:55591 (local address 127.0.0.1:43867)
0806 06:45:38.891785 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:38.891786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:38.891903 (+ 117us) server_negotiation.cc:407] Connection header received
0806 06:45:38.891973 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:38.891973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:38.891982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:38.892045 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:38.892542 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:38.893015 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:38.893694 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:38.893930 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:38.894007 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:38.895182 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:38.895183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:38.895184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:38.895190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:38.896591 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:38.896978 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:38.896979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:38.896979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:38.897037 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:38.897256 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:38.897257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:38.897258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:38.897364 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:38.897494 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:38.897613 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:45:38.897677 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:45:38.934376 4192 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:38.934906 4192 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:38.940908 4192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 592fb32b05dc48ae8538f1e71ed5a995: Generated new TSK 0
I20260806 06:45:38.941134 4192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:38.947630 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4109
W20260806 06:45:38.952704 4094 connection.cc:535] client connection to 127.0.0.1:43867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43867): Connection reset by peer (error 104)
2026-08-06T06:45:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:38 dist-test-slave-1xrh krb5kdc[4214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:39 dist-test-slave-1xrh krb5kdc[4214](info): set up 2 sockets
Aug 06 06:45:39 dist-test-slave-1xrh krb5kdc[4214](info): commencing operation
krb5kdc: starting...
W20260806 06:45:39.983374 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:40 dist-test-slave-1xrh krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:40.099917 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:40.185464 4228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:40.185761 4228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:40.185866 4228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:40.187965 4228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:40.188025 4228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:40.188051 4228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:40.188079 4228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:40.188100 4228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:40.190536 4228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44181
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:40.190848 4228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:40.191437 4228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:40.197371 4237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:40.197376 4234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:40.197424 4233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:40.197522 4228 server_base.cc:1034] running on GCE node
I20260806 06:45:40.197932 4228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:40.198304 4228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:40.199458 4228 hybrid_clock.cc:648] HybridClock initialized: now 1785998740199436 us; error 31 us; skew 500 ppm
Aug 06 06:45:40 dist-test-slave-1xrh krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998740, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:40.202955 4228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:40.203548 4228 webserver.cc:460] Webserver started at http://127.0.0.1:35191/ using document root <none> and password file <none>
I20260806 06:45:40.203773 4228 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:40.203845 4228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:40.203976 4228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:40.204979 4228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "b1223ffacaa94267bbfc5a0e75cc827e"
format_stamp: "Formatted at 2026-08-06 06:45:40 on dist-test-slave-1xrh"
server_key: "6b5b2f20a4f09463cb16710b52ab5a45"
server_key_iv: "bbf0d06db5cfdfb8e7ad03e1a714760a"
server_key_version: "encryptionkey@0"
I20260806 06:45:40.205353 4228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "b1223ffacaa94267bbfc5a0e75cc827e"
format_stamp: "Formatted at 2026-08-06 06:45:40 on dist-test-slave-1xrh"
server_key: "6b5b2f20a4f09463cb16710b52ab5a45"
server_key_iv: "bbf0d06db5cfdfb8e7ad03e1a714760a"
server_key_version: "encryptionkey@0"
I20260806 06:45:40.206748 4228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:40.207661 4244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:40.207864 4228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:45:40.207970 4228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "b1223ffacaa94267bbfc5a0e75cc827e"
format_stamp: "Formatted at 2026-08-06 06:45:40 on dist-test-slave-1xrh"
server_key: "6b5b2f20a4f09463cb16710b52ab5a45"
server_key_iv: "bbf0d06db5cfdfb8e7ad03e1a714760a"
server_key_version: "encryptionkey@0"
I20260806 06:45:40.208048 4228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:40.234472 4228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:40.236793 4228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:40.236965 4228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:40.241089 4228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44181
I20260806 06:45:40.241134 4304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44181 every 8 connection(s)
I20260806 06:45:40.241463 4228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:40.242373 4305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:40.244168 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4228
I20260806 06:45:40.244306 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:40.244474 14541 external_mini_cluster.cc:1428] Setting key 4171050a8edabe49e13c5b217881706f
I20260806 06:45:40.244912 4305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e: Bootstrap starting.
I20260806 06:45:40.245651 4305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:40.245966 4305 log.cc:824] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:40.246762 4305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e: No bootstrap required, opened a new log
I20260806 06:45:40.249003 4305 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } }
I20260806 06:45:40.249168 4305 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:40.249204 4305 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1223ffacaa94267bbfc5a0e75cc827e, State: Initialized, Role: FOLLOWER
I20260806 06:45:40.249346 4305 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } }
I20260806 06:45:40.249433 4305 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:40.249467 4305 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:40.249509 4305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:40.250224 4305 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } }
I20260806 06:45:40.250321 4305 leader_election.cc:302] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1223ffacaa94267bbfc5a0e75cc827e; no voters:
I20260806 06:45:40.250509 4305 leader_election.cc:288] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:40.250528 4311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:40.250771 4311 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [term 1 LEADER]: Becoming Leader. State: Replica: b1223ffacaa94267bbfc5a0e75cc827e, State: Running, Role: LEADER
I20260806 06:45:40.250779 4305 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:40.250885 4311 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } }
I20260806 06:45:40.250937 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:40.244987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:37689 (local address 127.0.0.1:44181)
0806 06:45:40.245171 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:40.245180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:40.245249 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:45:40.245876 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:40.245885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:40.245954 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:40.246099 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:40.246343 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:40.247822 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:40.248606 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:40.249238 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:40.249344 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:40.250136 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:40.250140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:40.250140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:40.250320 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:40.250364 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:40.250375 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:40.250471 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:40.250639 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:45:40.250668 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260806 06:45:40.251237 4313 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1223ffacaa94267bbfc5a0e75cc827e. Latest consensus state: current_term: 1 leader_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } } }
I20260806 06:45:40.251597 4313 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:40.251844 4320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:40.252059 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:40.252421 4320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:40.252408 4312 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1223ffacaa94267bbfc5a0e75cc827e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44181 } } }
I20260806 06:45:40.252767 4312 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:40.254020 4320 catalog_manager.cc:1347] Generated new cluster ID: aa7cab46eddb43669040ae36878762de
I20260806 06:45:40.254086 4320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:40.262471 4320 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:40.262715 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:40.255550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:36031 (local address 127.0.0.1:44181)
0806 06:45:40.255581 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:40.255582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:40.259313 (+ 3731us) server_negotiation.cc:407] Connection header received
0806 06:45:40.259379 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:40.259380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:40.259390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:40.259445 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:40.259573 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:40.260294 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:40.261326 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:40.261639 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:40.261703 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:40.262423 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:40.262424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:40.262424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:40.262428 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:40.262440 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:40.262444 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:40.262490 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:40.262613 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 06:45:40.262623 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:40.263211 4320 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:40.268364 4320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1223ffacaa94267bbfc5a0e75cc827e: Generated new TSK 0
I20260806 06:45:40.268520 4320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:40.271486 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4228
W20260806 06:45:40.277191 4211 connection.cc:535] client connection to 127.0.0.1:44181 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44181): Connection reset by peer (error 104)
2026-08-06T06:45:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:40 dist-test-slave-1xrh krb5kdc[4338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:40 dist-test-slave-1xrh krb5kdc[4338](info): set up 2 sockets
Aug 06 06:45:40 dist-test-slave-1xrh krb5kdc[4338](info): commencing operation
krb5kdc: starting...
W20260806 06:45:41.330334 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:41 dist-test-slave-1xrh krb5kdc[4338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:45:41.446710 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:41.526515 4351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:41.526710 4351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:41.526732 4351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:41.527808 4351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:41.527842 4351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:41.527856 4351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:41.527868 4351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:41.527880 4351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:41.529266 4351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:41.529448 4351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:41.529642 4351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:41.534453 4357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:41.534432 4359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:41.534499 4351 server_base.cc:1034] running on GCE node
W20260806 06:45:41.534463 4356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:41.534863 4351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:41.535187 4351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:41.536338 4351 hybrid_clock.cc:648] HybridClock initialized: now 1785998741536320 us; error 35 us; skew 500 ppm
Aug 06 06:45:41 dist-test-slave-1xrh krb5kdc[4338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998741, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:41.538744 4351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:45:41.539209 4351 webserver.cc:460] Webserver started at http://127.0.0.1:39133/ using document root <none> and password file <none>
I20260806 06:45:41.539445 4351 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:41.539520 4351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:41.539676 4351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:41.540664 4351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "7202ed65758a447993e2063c059ea8a0"
format_stamp: "Formatted at 2026-08-06 06:45:41 on dist-test-slave-1xrh"
server_key: "4d2839e58dcc989b5c8306b4431251c7"
server_key_iv: "46a84c7625d2c0613eacd1ec6850f4e2"
server_key_version: "encryptionkey@0"
I20260806 06:45:41.541013 4351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "7202ed65758a447993e2063c059ea8a0"
format_stamp: "Formatted at 2026-08-06 06:45:41 on dist-test-slave-1xrh"
server_key: "4d2839e58dcc989b5c8306b4431251c7"
server_key_iv: "46a84c7625d2c0613eacd1ec6850f4e2"
server_key_version: "encryptionkey@0"
I20260806 06:45:41.542256 4351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:41.542980 4368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:41.543171 4351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:41.543278 4351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "7202ed65758a447993e2063c059ea8a0"
format_stamp: "Formatted at 2026-08-06 06:45:41 on dist-test-slave-1xrh"
server_key: "4d2839e58dcc989b5c8306b4431251c7"
server_key_iv: "46a84c7625d2c0613eacd1ec6850f4e2"
server_key_version: "encryptionkey@0"
I20260806 06:45:41.543368 4351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:41.546455 4351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:41.548713 4351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:41.548866 4351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:41.552608 4351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39983
I20260806 06:45:41.552651 4426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39983 every 8 connection(s)
I20260806 06:45:41.552968 4351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:41.553797 4427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:41.555951 4427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0: Bootstrap starting.
I20260806 06:45:41.556493 4427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:41.556766 4427 log.cc:824] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:41.557411 4427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0: No bootstrap required, opened a new log
I20260806 06:45:41.558943 4427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } }
I20260806 06:45:41.559072 4427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:41.559146 4427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7202ed65758a447993e2063c059ea8a0, State: Initialized, Role: FOLLOWER
I20260806 06:45:41.559333 4427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } }
I20260806 06:45:41.559412 4427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:41.559453 4427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:41.559517 4427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:41.560030 4427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } }
I20260806 06:45:41.560137 4427 leader_election.cc:302] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7202ed65758a447993e2063c059ea8a0; no voters:
I20260806 06:45:41.560313 4427 leader_election.cc:288] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:41.560336 4431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:41.560518 4431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7202ed65758a447993e2063c059ea8a0, State: Running, Role: LEADER
I20260806 06:45:41.560560 4427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:41.560616 4431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } }
I20260806 06:45:41.560951 4433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7202ed65758a447993e2063c059ea8a0. Latest consensus state: current_term: 1 leader_uuid: "7202ed65758a447993e2063c059ea8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } } }
I20260806 06:45:41.561044 4433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:41.561061 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4351
I20260806 06:45:41.561192 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:41.561192 4432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7202ed65758a447993e2063c059ea8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7202ed65758a447993e2063c059ea8a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39983 } } }
I20260806 06:45:41.561266 4439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:41.561334 4432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:41.561409 14541 external_mini_cluster.cc:1428] Setting key 670213cfa7e6b2b176a92c9e69387bed
I20260806 06:45:41.561848 4439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:41.563606 4439 catalog_manager.cc:1347] Generated new cluster ID: ac0833e129134765ad1f10cfd83db3ff
I20260806 06:45:41.563661 4439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:41.563941 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:41.562041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:48481 (local address 127.0.0.1:39983)
0806 06:45:41.562206 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:41.562215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:41.562246 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:41.562527 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:41.562534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:41.562581 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:41.562720 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:41.563407 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:41.563410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:41.563410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:41.563415 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:41.563456 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:41.563470 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:41.563536 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:41.563706 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:45:41.563753 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260806 06:45:41.564774 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:41.567279 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:41.566091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:36779 (local address 127.0.0.1:39983)
0806 06:45:41.566121 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:41.566124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:41.566307 (+ 183us) server_negotiation.cc:407] Connection header received
0806 06:45:41.566362 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:41.566363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:41.566368 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:41.566423 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:41.567019 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:41.567020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:41.567020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:41.567024 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:41.567036 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:41.567041 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:41.567097 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:41.567187 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:45:41.567201 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:45:41.576843 4439 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:41.577016 4439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:41.583570 4439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7202ed65758a447993e2063c059ea8a0: Generated new TSK 0
I20260806 06:45:41.583734 4439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:41.587632 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4351
2026-08-06T06:45:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:41 dist-test-slave-1xrh krb5kdc[4462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:41 dist-test-slave-1xrh krb5kdc[4462](info): set up 2 sockets
Aug 06 06:45:41 dist-test-slave-1xrh krb5kdc[4462](info): commencing operation
krb5kdc: starting...
W20260806 06:45:42.635083 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:42.751667 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35651
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:42.833712 4475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:42.833912 4475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:42.833935 4475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:42.835070 4475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:42.835116 4475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:42.835196 4475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:42.835215 4475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:42.835244 4475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:42.836673 4475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:35651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:42.836920 4475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:42.837138 4475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:42.841854 4483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:42.841904 4480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:42.841928 4475 server_base.cc:1034] running on GCE node
W20260806 06:45:42.841907 4481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:42.842300 4475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:42.842592 4475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:42.843763 4475 hybrid_clock.cc:648] HybridClock initialized: now 1785998742843742 us; error 35 us; skew 500 ppm
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998742, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:42.846247 4475 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:42.846606 4475 webserver.cc:460] Webserver started at http://127.14.51.126:39865/ using document root <none> and password file <none>
I20260806 06:45:42.846778 4475 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:42.846817 4475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:42.846900 4475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:42.847924 4475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "5e1330414cfb40619523efb3dfa713ce"
format_stamp: "Formatted at 2026-08-06 06:45:42 on dist-test-slave-1xrh"
server_key: "f16fb86d1f51fe5517a5bb7779e07ecf"
server_key_iv: "4226ef3965271f2dd253b4b7ec316a3b"
server_key_version: "encryptionkey@0"
I20260806 06:45:42.848208 4475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "5e1330414cfb40619523efb3dfa713ce"
format_stamp: "Formatted at 2026-08-06 06:45:42 on dist-test-slave-1xrh"
server_key: "f16fb86d1f51fe5517a5bb7779e07ecf"
server_key_iv: "4226ef3965271f2dd253b4b7ec316a3b"
server_key_version: "encryptionkey@0"
I20260806 06:45:42.849408 4475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:42.850055 4490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:42.850203 4475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:42.850286 4475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "5e1330414cfb40619523efb3dfa713ce"
format_stamp: "Formatted at 2026-08-06 06:45:42 on dist-test-slave-1xrh"
server_key: "f16fb86d1f51fe5517a5bb7779e07ecf"
server_key_iv: "4226ef3965271f2dd253b4b7ec316a3b"
server_key_version: "encryptionkey@0"
I20260806 06:45:42.850339 4475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:42.873118 4475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:42.874873 4475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:42.874982 4475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:42.878778 4475 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:35651
I20260806 06:45:42.878837 4544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:35651 every 8 connection(s)
I20260806 06:45:42.879155 4475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:42.880017 4545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:42.882200 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce: Bootstrap starting.
I20260806 06:45:42.882754 4545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:42.883033 4545 log.cc:824] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:42.883790 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce: No bootstrap required, opened a new log
I20260806 06:45:42.885339 4545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } }
I20260806 06:45:42.885489 4545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:42.885569 4545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e1330414cfb40619523efb3dfa713ce, State: Initialized, Role: FOLLOWER
I20260806 06:45:42.885753 4545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } }
I20260806 06:45:42.885852 4545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:42.885895 4545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:42.885944 4545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:42.886426 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4475
I20260806 06:45:42.886516 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:42.886714 14541 external_mini_cluster.cc:1428] Setting key db459247357bd47f3d8f915d53ca54e5
I20260806 06:45:42.886863 4545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } }
I20260806 06:45:42.886973 4545 leader_election.cc:302] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e1330414cfb40619523efb3dfa713ce; no voters:
I20260806 06:45:42.887178 4545 leader_election.cc:288] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:42.887219 4549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:42.887431 4549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [term 1 LEADER]: Becoming Leader. State: Replica: 5e1330414cfb40619523efb3dfa713ce, State: Running, Role: LEADER
I20260806 06:45:42.887470 4545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:42.887583 4549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } }
I20260806 06:45:42.888062 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e1330414cfb40619523efb3dfa713ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } } }
I20260806 06:45:42.888142 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:42.888149 4553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e1330414cfb40619523efb3dfa713ce. Latest consensus state: current_term: 1 leader_uuid: "5e1330414cfb40619523efb3dfa713ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e1330414cfb40619523efb3dfa713ce" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35651 } } }
I20260806 06:45:42.888829 4553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:42.888989 4560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:42.889448 4560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:42.890758 4560 catalog_manager.cc:1347] Generated new cluster ID: 8c6d0ee40ee04dc5b7766455c93c5bbe
I20260806 06:45:42.890806 4560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:42.899083 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:42.887246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59749 (local address 127.14.51.126:35651)
0806 06:45:42.887633 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:42.887644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:42.887679 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:45:42.887825 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:42.887831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:42.887885 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:42.888031 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:42.888964 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:42.890379 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:42.891471 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:42.892060 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:42.892152 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:42.894907 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:42.894911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:42.894911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:42.894924 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:42.897429 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:42.897871 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:42.897874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:42.897875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:42.897991 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:42.898236 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:42.898237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:42.898237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:42.898550 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:42.898713 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:42.898791 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 06:45:42.898885 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":78,"threads_started":1}
I20260806 06:45:42.900068 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:42.901026 4560 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:42.901683 4560 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:42.907742 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:42.901577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45491 (local address 127.14.51.126:35651)
0806 06:45:42.901613 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:42.901614 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:42.901713 (+ 99us) server_negotiation.cc:407] Connection header received
0806 06:45:42.901810 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:42.901811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:42.901831 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:42.901893 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:42.902366 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:42.902843 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:42.903636 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:42.903890 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:42.903963 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:42.905129 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:42.905130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:42.905131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:42.905138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:42.906510 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:42.906881 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:42.906882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:42.906883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:42.906938 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:42.907226 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:42.907227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:42.907227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:42.907339 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:42.907463 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:42.907573 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:45:42.907635 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:45:42.912240 4560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e1330414cfb40619523efb3dfa713ce: Generated new TSK 0
I20260806 06:45:42.912405 4560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:42.917554 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4475
W20260806 06:45:42.922371 4460 connection.cc:535] client connection to 127.14.51.126:35651 recv error: Network error: failed to read from TLS socket (remote: 127.14.51.126:35651): Connection reset by peer (error 104)
2026-08-06T06:45:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4580](info): set up 2 sockets
Aug 06 06:45:42 dist-test-slave-1xrh krb5kdc[4580](info): commencing operation
krb5kdc: starting...
W20260806 06:45:43.981998 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:44 dist-test-slave-1xrh krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:44.095607 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44195
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:44.175293 4594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:44.175470 4594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:44.175493 4594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:44.176549 4594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:44.176587 4594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:44.176601 4594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:44.176635 4594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:44.176661 4594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:44.178046 4594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:44195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:44.178207 4594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:44.178412 4594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:45:44.183521 4594 server_base.cc:1034] running on GCE node
W20260806 06:45:44.183496 4599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:44.183490 4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:44.183480 4602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:44.183964 4594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:44.184192 4594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:44.185362 4594 hybrid_clock.cc:648] HybridClock initialized: now 1785998744185341 us; error 33 us; skew 500 ppm
Aug 06 06:45:44 dist-test-slave-1xrh krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998744, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:44.191011 4594 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:44.191497 4594 webserver.cc:460] Webserver started at http://127.14.51.126:37367/ using document root <none> and password file <none>
I20260806 06:45:44.191753 4594 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:44.191820 4594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:44.191974 4594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:44.192953 4594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "ca09ebbdacd643198b217fca05ed4d5c"
format_stamp: "Formatted at 2026-08-06 06:45:44 on dist-test-slave-1xrh"
server_key: "862bd13c28ec7bcf4b9ee24c84100ced"
server_key_iv: "9d9284502e69adf92b7e72cf918df2dd"
server_key_version: "encryptionkey@0"
I20260806 06:45:44.193300 4594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "ca09ebbdacd643198b217fca05ed4d5c"
format_stamp: "Formatted at 2026-08-06 06:45:44 on dist-test-slave-1xrh"
server_key: "862bd13c28ec7bcf4b9ee24c84100ced"
server_key_iv: "9d9284502e69adf92b7e72cf918df2dd"
server_key_version: "encryptionkey@0"
I20260806 06:45:44.194588 4594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:45:44.195312 4609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:44.195536 4594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:44.195626 4594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "ca09ebbdacd643198b217fca05ed4d5c"
format_stamp: "Formatted at 2026-08-06 06:45:44 on dist-test-slave-1xrh"
server_key: "862bd13c28ec7bcf4b9ee24c84100ced"
server_key_iv: "9d9284502e69adf92b7e72cf918df2dd"
server_key_version: "encryptionkey@0"
I20260806 06:45:44.195710 4594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:44.208595 4594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:44.210759 4594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:44.210907 4594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:44.214833 4594 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:44195
I20260806 06:45:44.214862 4661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:44195 every 8 connection(s)
I20260806 06:45:44.215250 4594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:44.216089 4662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:44.218170 4662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c: Bootstrap starting.
I20260806 06:45:44.218721 4662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:44.218997 4662 log.cc:824] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:44.219693 4662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c: No bootstrap required, opened a new log
I20260806 06:45:44.219764 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4594
I20260806 06:45:44.219904 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:44.220086 14541 external_mini_cluster.cc:1428] Setting key ac01fb1602c651e561b4c866ae3a26c7
I20260806 06:45:44.221629 4662 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } }
I20260806 06:45:44.221779 4662 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:44.221814 4662 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca09ebbdacd643198b217fca05ed4d5c, State: Initialized, Role: FOLLOWER
I20260806 06:45:44.221987 4662 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } }
I20260806 06:45:44.222078 4662 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:44.222146 4662 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:44.222190 4662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:44.222659 4662 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } }
I20260806 06:45:44.222761 4662 leader_election.cc:302] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca09ebbdacd643198b217fca05ed4d5c; no voters:
I20260806 06:45:44.222919 4662 leader_election.cc:288] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:44.222940 4667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:44.223049 4667 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [term 1 LEADER]: Becoming Leader. State: Replica: ca09ebbdacd643198b217fca05ed4d5c, State: Running, Role: LEADER
I20260806 06:45:44.223171 4662 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:44.223194 4667 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } }
I20260806 06:45:44.223620 4668 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca09ebbdacd643198b217fca05ed4d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } } }
I20260806 06:45:44.223623 4667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca09ebbdacd643198b217fca05ed4d5c. Latest consensus state: current_term: 1 leader_uuid: "ca09ebbdacd643198b217fca05ed4d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca09ebbdacd643198b217fca05ed4d5c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 44195 } } }
I20260806 06:45:44.223843 4667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:44.223834 4668 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:44.224556 4676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:44.225001 4676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:44.226603 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:44.220601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58483 (local address 127.14.51.126:44195)
0806 06:45:44.220790 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:44.220798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:44.220831 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:45:44.221618 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:44.221625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:44.221667 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:44.221770 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:44.222030 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:44.223593 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:44.224779 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:44.225235 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:44.225345 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:44.225968 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:44.225971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:44.225971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:44.226133 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:44.226160 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:44.226170 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:44.226260 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:44.226377 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:45:44.226407 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260806 06:45:44.227285 4676 catalog_manager.cc:1347] Generated new cluster ID: 23694f98434e46d28bc25d41ffec08a3
I20260806 06:45:44.227335 4676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:44.227717 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:44.231571 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:44.229001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57621 (local address 127.14.51.126:44195)
0806 06:45:44.229034 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:44.229036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:44.229174 (+ 138us) server_negotiation.cc:407] Connection header received
0806 06:45:44.229241 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:44.229241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:44.229251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:44.229292 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:44.229433 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:44.229864 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:44.230504 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:44.230716 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:44.230779 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:44.231354 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:44.231355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:44.231355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:44.231359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:44.231370 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:44.231373 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:44.231420 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:44.231475 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:45:44.231484 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:44.256213 4676 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:44.256880 4676 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:44.261698 4676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca09ebbdacd643198b217fca05ed4d5c: Generated new TSK 0
I20260806 06:45:44.261861 4676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:44.265269 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4594
W20260806 06:45:44.270671 4578 connection.cc:535] client connection to 127.14.51.126:44195 recv error: Network error: failed to read from TLS socket (remote: 127.14.51.126:44195): Connection reset by peer (error 104)
2026-08-06T06:45:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:44 dist-test-slave-1xrh krb5kdc[4695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:44 dist-test-slave-1xrh krb5kdc[4695](info): set up 2 sockets
Aug 06 06:45:44 dist-test-slave-1xrh krb5kdc[4695](info): commencing operation
krb5kdc: starting...
W20260806 06:45:45.306171 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:45 dist-test-slave-1xrh krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:45.422178 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:46843
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:46843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:45.501446 4709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:45.501669 4709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:45.501708 4709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:45.502816 4709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:45.502882 4709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:45.502907 4709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:45.502933 4709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:45.502964 4709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:45.504415 4709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:46843
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:46843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:45.504675 4709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:45.504884 4709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:45:45.509752 4709 server_base.cc:1034] running on GCE node
W20260806 06:45:45.509735 4716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:45.509735 4718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:45.509720 4715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:45.510226 4709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:45.510491 4709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:45.511631 4709 hybrid_clock.cc:648] HybridClock initialized: now 1785998745511614 us; error 40 us; skew 500 ppm
Aug 06 06:45:45 dist-test-slave-1xrh krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998745, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:45.514072 4709 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:45.514503 4709 webserver.cc:460] Webserver started at http://127.14.51.126:39475/ using document root <none> and password file <none>
I20260806 06:45:45.514714 4709 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:45.514756 4709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:45.514916 4709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:45.516031 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "d2f68738c4824081a33b77f301fa6732"
format_stamp: "Formatted at 2026-08-06 06:45:45 on dist-test-slave-1xrh"
server_key: "17529df27fc80d105c939df10be92058"
server_key_iv: "0ee7e0cf2c977d95adcf51b25437b323"
server_key_version: "encryptionkey@0"
I20260806 06:45:45.516381 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "d2f68738c4824081a33b77f301fa6732"
format_stamp: "Formatted at 2026-08-06 06:45:45 on dist-test-slave-1xrh"
server_key: "17529df27fc80d105c939df10be92058"
server_key_iv: "0ee7e0cf2c977d95adcf51b25437b323"
server_key_version: "encryptionkey@0"
I20260806 06:45:45.517673 4709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:45.518355 4725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:45.518522 4709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:45.518620 4709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "d2f68738c4824081a33b77f301fa6732"
format_stamp: "Formatted at 2026-08-06 06:45:45 on dist-test-slave-1xrh"
server_key: "17529df27fc80d105c939df10be92058"
server_key_iv: "0ee7e0cf2c977d95adcf51b25437b323"
server_key_version: "encryptionkey@0"
I20260806 06:45:45.518700 4709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:45.521894 4709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:45.523744 4709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:45.523907 4709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:45.527756 4709 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:46843
I20260806 06:45:45.527789 4779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:46843 every 8 connection(s)
I20260806 06:45:45.528093 4709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:45.528973 4780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:45.531083 4780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732: Bootstrap starting.
I20260806 06:45:45.531769 4780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:45.532073 4780 log.cc:824] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:45.532747 4780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732: No bootstrap required, opened a new log
I20260806 06:45:45.534337 4780 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } }
I20260806 06:45:45.534468 4780 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:45.534515 4780 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f68738c4824081a33b77f301fa6732, State: Initialized, Role: FOLLOWER
I20260806 06:45:45.534684 4780 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } }
I20260806 06:45:45.534750 4780 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:45.534791 4780 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:45.534852 4780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:45.535748 4780 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } }
I20260806 06:45:45.535866 4780 leader_election.cc:302] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f68738c4824081a33b77f301fa6732; no voters:
I20260806 06:45:45.536067 4780 leader_election.cc:288] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:45.536118 4783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:45.536132 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4709
I20260806 06:45:45.536237 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:45.536273 4783 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [term 1 LEADER]: Becoming Leader. State: Replica: d2f68738c4824081a33b77f301fa6732, State: Running, Role: LEADER
I20260806 06:45:45.536322 4780 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:45.536351 4783 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } }
I20260806 06:45:45.536424 14541 external_mini_cluster.cc:1428] Setting key 3d78b7d855e2273a76b9b7db21c30a72
I20260806 06:45:45.536726 4783 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f68738c4824081a33b77f301fa6732. Latest consensus state: current_term: 1 leader_uuid: "d2f68738c4824081a33b77f301fa6732" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } } }
I20260806 06:45:45.536734 4784 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f68738c4824081a33b77f301fa6732" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f68738c4824081a33b77f301fa6732" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46843 } } }
I20260806 06:45:45.536911 4784 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:45.537144 4783 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:45.537498 4793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:45.537986 4793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:45.539253 4794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:45.537118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54685 (local address 127.14.51.126:46843)
0806 06:45:45.537485 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:45.537494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:45.537553 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:45:45.537658 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:45.537664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:45.537695 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:45.537838 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:45.538770 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:45.538773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:45.538774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:45.538778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:45.538835 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:45.538850 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:45.538932 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:45.539008 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:45:45.539047 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":64,"threads_started":1}
I20260806 06:45:45.539925 4793 catalog_manager.cc:1347] Generated new cluster ID: 94e645642cb44014a8a2c889628409a5
I20260806 06:45:45.539983 4793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:45.540239 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:45.544588 4794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:45.543471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57337 (local address 127.14.51.126:46843)
0806 06:45:45.543509 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:45.543511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:45.543688 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:45:45.543729 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:45.543730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:45.543734 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:45.543817 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:45.544359 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:45.544360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:45.544360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:45.544365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:45.544379 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:45.544384 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:45.544446 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:45.544505 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:45:45.544518 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:45:45.559794 4793 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:45.559945 4793 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:45.566208 4793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f68738c4824081a33b77f301fa6732: Generated new TSK 0
I20260806 06:45:45.566390 4793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:45.570736 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4709
2026-08-06T06:45:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:45 dist-test-slave-1xrh krb5kdc[4812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:45 dist-test-slave-1xrh krb5kdc[4812](info): set up 2 sockets
Aug 06 06:45:45 dist-test-slave-1xrh krb5kdc[4812](info): commencing operation
krb5kdc: starting...
W20260806 06:45:46.673898 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:46.785976 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:46775
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:37565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:46775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:46.865586 4826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:46.865806 4826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:46.865857 4826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:46.867012 4826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:46.867075 4826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:46.867101 4826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:46.867166 4826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:46.867195 4826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:46.868732 4826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:37565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:46775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:46775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:46.868999 4826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:46.869212 4826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:46.873965 4835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:46.873965 4833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:46.873993 4832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:46.874164 4826 server_base.cc:1034] running on GCE node
I20260806 06:45:46.874478 4826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:46.874716 4826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:46.875862 4826 hybrid_clock.cc:648] HybridClock initialized: now 1785998746875849 us; error 32 us; skew 500 ppm
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998746, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:46.878286 4826 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:46.878692 4826 webserver.cc:460] Webserver started at http://127.14.51.126:35345/ using document root <none> and password file <none>
I20260806 06:45:46.878871 4826 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:46.878911 4826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:46.878993 4826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:46.880026 4826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "207175e72eca4ca987783005d851aa14"
format_stamp: "Formatted at 2026-08-06 06:45:46 on dist-test-slave-1xrh"
server_key: "0c6159a3b32b8adc5c34a7d05b6d59cf"
server_key_iv: "8c09db009cb4d617dbc1f230dc631b00"
server_key_version: "encryptionkey@0"
I20260806 06:45:46.880405 4826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "207175e72eca4ca987783005d851aa14"
format_stamp: "Formatted at 2026-08-06 06:45:46 on dist-test-slave-1xrh"
server_key: "0c6159a3b32b8adc5c34a7d05b6d59cf"
server_key_iv: "8c09db009cb4d617dbc1f230dc631b00"
server_key_version: "encryptionkey@0"
I20260806 06:45:46.881614 4826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:46.882237 4842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:46.882402 4826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:46.882504 4826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "207175e72eca4ca987783005d851aa14"
format_stamp: "Formatted at 2026-08-06 06:45:46 on dist-test-slave-1xrh"
server_key: "0c6159a3b32b8adc5c34a7d05b6d59cf"
server_key_iv: "8c09db009cb4d617dbc1f230dc631b00"
server_key_version: "encryptionkey@0"
I20260806 06:45:46.882588 4826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:46.895009 4826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:46.896724 4826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:46.896874 4826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:46.900703 4826 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:46775
I20260806 06:45:46.900738 4899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:46775 every 8 connection(s)
I20260806 06:45:46.901064 4826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:46.901837 4900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:46.903937 4900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14: Bootstrap starting.
I20260806 06:45:46.904424 4900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:46.904662 4900 log.cc:824] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:46.905292 4900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14: No bootstrap required, opened a new log
I20260806 06:45:46.906826 4900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } }
I20260806 06:45:46.906975 4900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:46.907018 4900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207175e72eca4ca987783005d851aa14, State: Initialized, Role: FOLLOWER
I20260806 06:45:46.907199 4900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } }
I20260806 06:45:46.907282 4900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:46.907326 4900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:46.907377 4900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:46.908164 4900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } }
I20260806 06:45:46.908279 4900 leader_election.cc:302] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 207175e72eca4ca987783005d851aa14; no voters:
I20260806 06:45:46.908456 4900 leader_election.cc:288] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:46.908500 4903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:46.908661 4903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [term 1 LEADER]: Becoming Leader. State: Replica: 207175e72eca4ca987783005d851aa14, State: Running, Role: LEADER
I20260806 06:45:46.908713 4900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:46.908800 4903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } }
I20260806 06:45:46.909225 4904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "207175e72eca4ca987783005d851aa14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } } }
I20260806 06:45:46.909267 4905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207175e72eca4ca987783005d851aa14. Latest consensus state: current_term: 1 leader_uuid: "207175e72eca4ca987783005d851aa14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207175e72eca4ca987783005d851aa14" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 46775 } } }
I20260806 06:45:46.909319 4904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:46.909343 4905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:46.909512 4912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:46.910105 4912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:46.910182 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4826
I20260806 06:45:46.910306 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:46.910533 14541 external_mini_cluster.cc:1428] Setting key 264b73899901a0f6761e8dfa714773e5
I20260806 06:45:46.911635 4912 catalog_manager.cc:1347] Generated new cluster ID: b6391646a4354e728e22fa8865eaf004
I20260806 06:45:46.911692 4912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:46.920010 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:46.910993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57799 (local address 127.14.51.126:46775)
0806 06:45:46.911166 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:46.911174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:46.911328 (+ 154us) server_negotiation.cc:407] Connection header received
0806 06:45:46.911439 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:46.911445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:46.911523 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:46.911666 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:46.912349 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:46.913242 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:46.914016 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:46.914427 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:46.914542 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:46.916906 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:46.916910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:46.916910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:46.916920 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:46.918499 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:46.918887 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:46.918890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:46.918890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:46.918986 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:46.919202 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:46.919203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:46.919204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:46.919419 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:46.919603 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:46.919689 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:45:46.919828 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260806 06:45:46.920943 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:46.924536 4912 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:46.925200 4912 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:46.934381 4912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 207175e72eca4ca987783005d851aa14: Generated new TSK 0
I20260806 06:45:46.934541 4912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:46.937428 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:46.922201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36681 (local address 127.14.51.126:46775)
0806 06:45:46.922241 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:46.922243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:46.924361 (+ 2118us) server_negotiation.cc:407] Connection header received
0806 06:45:46.924380 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:46.924381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:46.924390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:46.924432 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:46.931766 (+ 7334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:46.932229 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:46.932918 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:46.933128 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:46.933201 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:46.934877 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:46.934882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:46.934882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:46.934888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:46.936318 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:46.936693 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:46.936694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:46.936694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:46.936749 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:46.936950 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:46.936951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:46.936951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:46.937052 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:46.937161 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:46.937256 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:45:46.937336 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:45:46.939211 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4826
2026-08-06T06:45:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4931](info): set up 2 sockets
Aug 06 06:45:46 dist-test-slave-1xrh krb5kdc[4931](info): commencing operation
krb5kdc: starting...
W20260806 06:45:47.978391 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:48 dist-test-slave-1xrh krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:48.091989 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35509
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:35509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:48.168773 4945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:48.168968 4945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:48.168999 4945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:48.170166 4945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:48.170210 4945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:48.170225 4945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:48.170238 4945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:48.170250 4945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:48.171741 4945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:35509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:35509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:48.172003 4945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:48.172242 4945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:48.176994 4950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:48.177000 4953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:48.177055 4951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:48.177233 4945 server_base.cc:1034] running on GCE node
I20260806 06:45:48.177440 4945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:48.177717 4945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:48.178872 4945 hybrid_clock.cc:648] HybridClock initialized: now 1785998748178852 us; error 38 us; skew 500 ppm
Aug 06 06:45:48 dist-test-slave-1xrh krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998748, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:48.181313 4945 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:48.181739 4945 webserver.cc:460] Webserver started at http://127.14.51.126:35777/ using document root <none> and password file <none>
I20260806 06:45:48.181990 4945 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:48.182039 4945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:48.182193 4945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:48.183215 4945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "dabd774ef0034b17a8a9b770f5407ece"
format_stamp: "Formatted at 2026-08-06 06:45:48 on dist-test-slave-1xrh"
server_key: "8ed1470748fd618de374066346410ec7"
server_key_iv: "eadc288eae35193005295f48d31a6261"
server_key_version: "encryptionkey@0"
I20260806 06:45:48.183561 4945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "dabd774ef0034b17a8a9b770f5407ece"
format_stamp: "Formatted at 2026-08-06 06:45:48 on dist-test-slave-1xrh"
server_key: "8ed1470748fd618de374066346410ec7"
server_key_iv: "eadc288eae35193005295f48d31a6261"
server_key_version: "encryptionkey@0"
I20260806 06:45:48.184844 4945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:48.185540 4960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:48.185693 4945 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:45:48.185804 4945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "dabd774ef0034b17a8a9b770f5407ece"
format_stamp: "Formatted at 2026-08-06 06:45:48 on dist-test-slave-1xrh"
server_key: "8ed1470748fd618de374066346410ec7"
server_key_iv: "eadc288eae35193005295f48d31a6261"
server_key_version: "encryptionkey@0"
I20260806 06:45:48.185885 4945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:48.209721 4945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:48.211995 4945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:48.212152 4945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:48.215939 4945 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:35509
I20260806 06:45:48.215970 5017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:35509 every 8 connection(s)
I20260806 06:45:48.216302 4945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:48.217089 5018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:48.219215 5018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece: Bootstrap starting.
I20260806 06:45:48.219769 5018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:48.220062 5018 log.cc:824] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:48.220719 5018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece: No bootstrap required, opened a new log
I20260806 06:45:48.222218 5018 raft_consensus.cc:348] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } }
I20260806 06:45:48.222365 5018 raft_consensus.cc:374] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:48.222421 5018 raft_consensus.cc:729] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabd774ef0034b17a8a9b770f5407ece, State: Initialized, Role: FOLLOWER
I20260806 06:45:48.222626 5018 consensus_queue.cc:260] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } }
I20260806 06:45:48.222713 5018 raft_consensus.cc:388] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:48.222770 5018 raft_consensus.cc:482] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:48.222826 5018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:48.223697 5018 raft_consensus.cc:504] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } }
I20260806 06:45:48.223820 5018 leader_election.cc:302] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dabd774ef0034b17a8a9b770f5407ece; no voters:
I20260806 06:45:48.223996 5018 leader_election.cc:288] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:48.224027 5021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:48.224179 5021 raft_consensus.cc:686] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [term 1 LEADER]: Becoming Leader. State: Replica: dabd774ef0034b17a8a9b770f5407ece, State: Running, Role: LEADER
I20260806 06:45:48.224241 5018 sys_catalog.cc:564] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:48.224308 5021 consensus_queue.cc:237] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } }
I20260806 06:45:48.224653 5022 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dabd774ef0034b17a8a9b770f5407ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } } }
I20260806 06:45:48.224663 5023 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader dabd774ef0034b17a8a9b770f5407ece. Latest consensus state: current_term: 1 leader_uuid: "dabd774ef0034b17a8a9b770f5407ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd774ef0034b17a8a9b770f5407ece" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 35509 } } }
I20260806 06:45:48.224789 5022 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:48.224799 5023 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:48.225051 5030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:48.225610 5030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:48.225961 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 4945
I20260806 06:45:48.226109 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:48.226300 14541 external_mini_cluster.cc:1428] Setting key a4fb6d2d62d74ba7c95e2c496c6b24ed
I20260806 06:45:48.227383 5030 catalog_manager.cc:1347] Generated new cluster ID: ab9c2873c2d549c8bf2eeb7940220458
I20260806 06:45:48.227435 5030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:48.232403 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:48.226888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42983 (local address 127.14.51.126:35509)
0806 06:45:48.227073 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:48.227081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:48.227112 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:48.227261 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:48.227268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:48.227321 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:48.227455 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:48.227761 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:48.229112 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:48.230166 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:48.230761 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:48.230848 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:48.231665 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:48.231669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:48.231669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:48.231845 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:48.231878 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:48.231890 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:48.231935 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:48.232171 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 06:45:48.232224 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260806 06:45:48.233249 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:48.235765 5030 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:48.236245 5030 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:48.237861 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:48.234690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49425 (local address 127.14.51.126:35509)
0806 06:45:48.234720 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:48.234722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:48.234729 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:48.234788 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:48.234788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:48.234796 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:48.234848 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:48.234977 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:48.235639 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:48.236647 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:48.236928 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:48.236993 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:48.237609 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:48.237610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:48.237611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:48.237614 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:48.237627 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:48.237631 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:48.237691 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:48.237763 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:45:48.237795 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:48.244352 5030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dabd774ef0034b17a8a9b770f5407ece: Generated new TSK 0
I20260806 06:45:48.244508 5030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:48.247699 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 4945
2026-08-06T06:45:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:48 dist-test-slave-1xrh krb5kdc[5049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:48 dist-test-slave-1xrh krb5kdc[5049](info): set up 2 sockets
Aug 06 06:45:48 dist-test-slave-1xrh krb5kdc[5049](info): commencing operation
krb5kdc: starting...
W20260806 06:45:49.277433 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:49 dist-test-slave-1xrh krb5kdc[5049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:49.401443 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38853
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:38853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:49.480701 5065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:49.480892 5065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:49.480933 5065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:49.482062 5065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:49.482102 5065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:49.482127 5065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:49.482139 5065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:49.482187 5065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:49.483637 5065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:38853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:38853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:49.483891 5065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:49.484102 5065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:49.488726 5076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:49.488797 5074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:49.488916 5065 server_base.cc:1034] running on GCE node
W20260806 06:45:49.489041 5071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:49.489284 5065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:49.489507 5065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:49.490649 5065 hybrid_clock.cc:648] HybridClock initialized: now 1785998749490621 us; error 45 us; skew 500 ppm
Aug 06 06:45:49 dist-test-slave-1xrh krb5kdc[5049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:49.498522 5065 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:49.498987 5065 webserver.cc:460] Webserver started at http://127.14.51.126:37667/ using document root <none> and password file <none>
I20260806 06:45:49.499259 5065 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:49.499328 5065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:49.499465 5065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:49.500507 5065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "5fea1d4e9abd42729787eaef44e4ac9c"
format_stamp: "Formatted at 2026-08-06 06:45:49 on dist-test-slave-1xrh"
server_key: "ffc798ae4bf38eb5a7e30a6c78cf1e2b"
server_key_iv: "c749eec13bc90597476cec51b2b82300"
server_key_version: "encryptionkey@0"
I20260806 06:45:49.500864 5065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "5fea1d4e9abd42729787eaef44e4ac9c"
format_stamp: "Formatted at 2026-08-06 06:45:49 on dist-test-slave-1xrh"
server_key: "ffc798ae4bf38eb5a7e30a6c78cf1e2b"
server_key_iv: "c749eec13bc90597476cec51b2b82300"
server_key_version: "encryptionkey@0"
I20260806 06:45:49.502178 5065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:45:49.502880 5083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:49.503057 5065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:49.503173 5065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "5fea1d4e9abd42729787eaef44e4ac9c"
format_stamp: "Formatted at 2026-08-06 06:45:49 on dist-test-slave-1xrh"
server_key: "ffc798ae4bf38eb5a7e30a6c78cf1e2b"
server_key_iv: "c749eec13bc90597476cec51b2b82300"
server_key_version: "encryptionkey@0"
I20260806 06:45:49.503273 5065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:49.506444 5065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:49.508311 5065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:49.508471 5065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:49.512444 5065 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:38853
I20260806 06:45:49.512476 5139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:38853 every 8 connection(s)
I20260806 06:45:49.512791 5065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:49.513713 5140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:49.515643 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5065
I20260806 06:45:49.515777 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:49.515954 14541 external_mini_cluster.cc:1428] Setting key d5edb28461d9a49f8dc9204652e53401
I20260806 06:45:49.516161 5140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c: Bootstrap starting.
I20260806 06:45:49.516909 5140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:49.517256 5140 log.cc:824] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:49.518345 5140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c: No bootstrap required, opened a new log
I20260806 06:45:49.518558 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:49.516461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56955 (local address 127.14.51.126:38853)
0806 06:45:49.516671 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:49.516679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:49.516712 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:45:49.517355 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:49.517363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:49.517396 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:49.517558 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:49.518124 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:49.518128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:49.518129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:49.518133 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:49.518177 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:49.518189 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:49.518280 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:49.518311 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:45:49.518351 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260806 06:45:49.519622 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:49.520597 5140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } }
I20260806 06:45:49.520749 5140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:45:49.520800 5140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fea1d4e9abd42729787eaef44e4ac9c, State: Initialized, Role: FOLLOWER
I20260806 06:45:49.520924 5140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } }
I20260806 06:45:49.521023 5140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:45:49.521059 5140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:45:49.521173 5140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:49.521698 5140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } }
I20260806 06:45:49.521813 5140 leader_election.cc:302] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fea1d4e9abd42729787eaef44e4ac9c; no voters:
I20260806 06:45:49.521971 5140 leader_election.cc:288] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:45:49.522002 5150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:49.522125 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:49.521044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45401 (local address 127.14.51.126:38853)
0806 06:45:49.521091 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:49.521092 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:49.521126 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:45:49.521201 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:49.521206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:49.521210 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:49.521279 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:49.521852 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:49.521853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:49.521853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:45:49.521857 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:49.521868 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:45:49.521871 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:49.521924 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:49.521998 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:45:49.522011 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:45:49.522181 5150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [term 1 LEADER]: Becoming Leader. State: Replica: 5fea1d4e9abd42729787eaef44e4ac9c, State: Running, Role: LEADER
I20260806 06:45:49.522256 5140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:49.522270 5150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } }
I20260806 06:45:49.522603 5151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } } }
I20260806 06:45:49.522601 5152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fea1d4e9abd42729787eaef44e4ac9c. Latest consensus state: current_term: 1 leader_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fea1d4e9abd42729787eaef44e4ac9c" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 38853 } } }
I20260806 06:45:49.522828 5152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:49.522995 5151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:49.523077 5159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:49.523607 5159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:49.525080 5159 catalog_manager.cc:1347] Generated new cluster ID: 088c40c4bf964d66b50280ff84a68516
I20260806 06:45:49.525133 5159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:49.558784 5159 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:49.558990 5159 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:49.567800 5159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fea1d4e9abd42729787eaef44e4ac9c: Generated new TSK 0
I20260806 06:45:49.567996 5159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:49.571164 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5065
2026-08-06T06:45:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1323 ms)
[----------] 18 tests from AuthTokenIssuingTest (23972 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:49 dist-test-slave-1xrh krb5kdc[5173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:49 dist-test-slave-1xrh krb5kdc[5173](info): set up 2 sockets
Aug 06 06:45:49 dist-test-slave-1xrh krb5kdc[5173](info): commencing operation
krb5kdc: starting...
W20260806 06:45:50.631065 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:50 dist-test-slave-1xrh krb5kdc[5173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:50.749881 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33399
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:50.830255 5186 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:50.830499 5186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:50.830533 5186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:50.830578 5186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:50.831825 5186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:50.831885 5186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:50.831908 5186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:50.831935 5186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:50.831966 5186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:50.833431 5186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:33399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:50.833710 5186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:50.833940 5186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:50.838749 5194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:50.838773 5191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:50.838914 5186 server_base.cc:1034] running on GCE node
W20260806 06:45:50.838987 5192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:50.839278 5186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:50.839583 5186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:50.840727 5186 hybrid_clock.cc:648] HybridClock initialized: now 1785998750840707 us; error 37 us; skew 500 ppm
Aug 06 06:45:50 dist-test-slave-1xrh krb5kdc[5173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:50.843200 5186 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:50.843652 5186 webserver.cc:460] Webserver started at http://127.14.51.126:34771/ using document root <none> and password file <none>
I20260806 06:45:50.843854 5186 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:50.843896 5186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:50.843990 5186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:50.845022 5186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "cb66ba90ddec4f93bfb462ed76a5ddac"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "df3e5e8a633eee8e80d01496757bd3e9"
server_key_iv: "8341c6f3fbe720aae8129b0dbdb1aba9"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.845327 5186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "cb66ba90ddec4f93bfb462ed76a5ddac"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "df3e5e8a633eee8e80d01496757bd3e9"
server_key_iv: "8341c6f3fbe720aae8129b0dbdb1aba9"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.846611 5186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:45:50.847321 5201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:50.847491 5186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:50.847602 5186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "cb66ba90ddec4f93bfb462ed76a5ddac"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "df3e5e8a633eee8e80d01496757bd3e9"
server_key_iv: "8341c6f3fbe720aae8129b0dbdb1aba9"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.847687 5186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:50.867358 5186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:50.869313 5186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:50.869464 5186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:50.873823 5186 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:33399
I20260806 06:45:50.873867 5257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:33399 every 8 connection(s)
I20260806 06:45:50.874253 5186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:50.874419 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5186
I20260806 06:45:50.874567 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:50.874805 14541 external_mini_cluster.cc:1428] Setting key f51474a04914c4a4aafa3ebc5f51f9c3
I20260806 06:45:50.875473 5258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:50.877000 5258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.51.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.125@KRBTEST.COM" created.
Aug 06 06:45:50 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:50.890081 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.877785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.14.51.126:33399)
0806 06:45:50.877923 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:50.877933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:50.878054 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:45:50.878880 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:50.878882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:50.878897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:50.878965 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:50.879817 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:50.880618 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:50.881959 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:50.882255 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:50.882331 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:50.886920 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:50.886928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:50.886928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:50.886934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:50.888490 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:50.888886 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:50.888889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:50.888889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:50.888969 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:50.889211 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:50.889212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:50.889213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:50.889414 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:50.889505 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:50.889706 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 06:45:50.889841 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260806 06:45:50.890084 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.877713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33399 (local address 127.0.0.1:48064)
0806 06:45:50.877923 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:50.877967 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:50.878710 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:50.879018 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:50.879021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:50.879448 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:50.879704 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:50.879711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:50.880720 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:50.880721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:50.881872 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:50.881876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:50.882366 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:50.882368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:50.882458 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:50.882997 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:50.883008 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:50.886818 (+ 3810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:50.888592 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:50.888599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:50.888601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:50.888789 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:50.889054 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:50.889055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:50.889056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:50.889136 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:50.889484 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:50.889489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:50.889562 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 06:45:50.889674 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:45:50.889904 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":95,"thread_start_us":117,"threads_started":1}
I20260806 06:45:50.890766 5258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:50.891168 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.890998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45550)
0806 06:45:50.891057 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:50.891114 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:45:50.891321 5204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
I20260806 06:45:50.891778 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.891659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45552)
0806 06:45:50.891719 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:50.891753 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 06:45:50.891901 5258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:41929: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
I20260806 06:45:50.902570 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:41929
--webserver_interface=127.14.51.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:50.941146 5258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } attempt: 1
I20260806 06:45:50.941684 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.941542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45554)
0806 06:45:50.941603 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:50.941651 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:45:50.942080 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:50.941956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45566)
0806 06:45:50.942028 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:50.942058 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 06:45:50.942209 5258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:41929: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
W20260806 06:45:50.982969 5266 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:50.983232 5266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:50.983274 5266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:50.983314 5266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:50.984429 5266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:50.984468 5266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:50.984510 5266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:50.984539 5266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:50.984558 5266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:50.985997 5266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.51.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:50.986225 5266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:50.986443 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:50.991109 5271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:50.991246 5272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:50.991204 5274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:50.991472 5266 server_base.cc:1034] running on GCE node
I20260806 06:45:50.991673 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:50.991897 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:50.993043 5266 hybrid_clock.cc:648] HybridClock initialized: now 1785998750993030 us; error 27 us; skew 500 ppm
Aug 06 06:45:50 dist-test-slave-1xrh krb5kdc[5173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:50.995414 5266 init.cc:377] Logged in from keytab as kudu/127.14.51.125@KRBTEST.COM (short username kudu)
I20260806 06:45:50.995766 5266 webserver.cc:460] Webserver started at http://127.14.51.125:43185/ using document root <none> and password file <none>
I20260806 06:45:50.995950 5266 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:50.995990 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:50.996083 5266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:50.997064 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data/instance:
uuid: "f5828198a5f2404cb72ad569cb165623"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "58e4cdb9d07b082d3875f114b46d874d"
server_key_iv: "2df2cf3bcc819a22db403deb0be32b0f"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.997352 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal/instance:
uuid: "f5828198a5f2404cb72ad569cb165623"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "58e4cdb9d07b082d3875f114b46d874d"
server_key_iv: "2df2cf3bcc819a22db403deb0be32b0f"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.998540 5266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:50.999225 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:50.999398 5266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:50.999495 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "f5828198a5f2404cb72ad569cb165623"
format_stamp: "Formatted at 2026-08-06 06:45:50 on dist-test-slave-1xrh"
server_key: "58e4cdb9d07b082d3875f114b46d874d"
server_key_iv: "2df2cf3bcc819a22db403deb0be32b0f"
server_key_version: "encryptionkey@0"
I20260806 06:45:50.999574 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:51.010456 5258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } attempt: 2
I20260806 06:45:51.010900 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.010766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45572)
0806 06:45:51.010833 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.010872 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 06:45:51.011315 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.011212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45580)
0806 06:45:51.011259 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.011287 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:51.011442 5258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:41929: Network error: Client connection negotiation failed: client connection to 127.14.51.125:41929: connect: Connection refused (error 111)
I20260806 06:45:51.016765 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:51.018556 5266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:51.018708 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:51.022503 5266 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.125:41929
I20260806 06:45:51.022555 5334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.125:41929 every 8 connection(s)
I20260806 06:45:51.022933 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
I20260806 06:45:51.023948 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:51.025354 5335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:51.027217 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5266
I20260806 06:45:51.027298 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-1/wal/instance
I20260806 06:45:51.027508 14541 external_mini_cluster.cc:1428] Setting key 72cee793fa512207125fdb3e9e47ad67
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.51.124@KRBTEST.COM; defaulting to no policy
I20260806 06:45:51.037454 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.025991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.14.51.126:33399)
0806 06:45:51.026049 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.026051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.026246 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:45:51.026670 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.026671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.026681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.026735 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.027405 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.028117 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.029015 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.029218 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.029303 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.031457 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.031458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.031459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.031466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.035672 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.036210 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.036211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.036212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.036280 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.036587 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.036587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.036588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.036744 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.036828 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.037218 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.037355 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.124@KRBTEST.COM" created.
I20260806 06:45:51.037869 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.025956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33399 (local address 127.0.0.1:48068)
0806 06:45:51.026133 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.026170 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.026596 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.026791 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.026794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.027010 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.027305 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.027312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.028204 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.028205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.028916 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.028918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.029336 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.029337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.029455 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.029937 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.029944 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.031361 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.035777 (+ 4416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.035785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.035787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.036108 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.036369 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.036370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.036371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.036517 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.036850 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.036858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.037053 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.037175 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.037396 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260806 06:45:51.039503 5335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:51.051188 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.039761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45584)
0806 06:45:51.039799 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.039812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.039857 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.043180 (+ 3323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.043181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.043344 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.043404 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.043406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.044766 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.044767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.045289 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.045291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.046017 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.046018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.046098 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.046486 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.046491 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.047817 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.049989 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.049990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.049991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.050228 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.050474 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.050475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.050475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.050536 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.050817 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.050818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.050856 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.050922 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.051034 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:45:51.051251 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.039808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.14.51.125:41929)
0806 06:45:51.042973 (+ 3165us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.042977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.043001 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:45:51.043022 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.043023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.043036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.043133 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.043490 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.044654 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.045391 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.045913 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.046011 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.047907 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.047915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.047915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.047922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.049896 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.050337 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.050340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.050340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.050404 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.050620 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.050621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.050621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.050740 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.050824 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.050958 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.051087 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":3134,"thread_start_us":69,"threads_started":1}
I20260806 06:45:51.051913 5335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:51.052232 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.052118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58444)
0806 06:45:51.052152 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.052201 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 06:45:51.052381 5282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
I20260806 06:45:51.052819 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.052720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58454)
0806 06:45:51.052763 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.052791 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:51.052942 5335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:40085: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
I20260806 06:45:51.053759 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:40085
--webserver_interface=127.14.51.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:51.102152 5335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } attempt: 1
I20260806 06:45:51.102635 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.102470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58460)
0806 06:45:51.102532 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.102576 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:45:51.102622 5258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } attempt: 3
I20260806 06:45:51.103152 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.103016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58470)
0806 06:45:51.103077 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.103105 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 06:45:51.103327 5335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:40085: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:51.109623 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.103044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.14.51.125:41929)
0806 06:45:51.103099 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.103101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.103110 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:51.103161 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.103162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.103175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.103237 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.103639 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.104135 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.104838 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.105058 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.105146 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.107040 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.107041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.107042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.107048 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.108511 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.108879 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.108880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.108881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.108938 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.109133 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.109134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.109135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.109245 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.109318 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.109466 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.109543 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:45:51.109640 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.102942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45596)
0806 06:45:51.103000 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.103011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.103055 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.103294 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.103295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.103486 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.103551 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.103552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.104222 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.104223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.104754 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.104755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.105146 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.105147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.105225 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.105691 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.105697 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.106910 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.108614 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.108615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.108616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.108795 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.109004 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.109005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.109005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.109062 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.109311 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.109312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.109358 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.109442 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.109551 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:45:51.110255 5258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:51.110606 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.110492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58484)
0806 06:45:51.110556 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.110582 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 06:45:51.110935 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.110847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58494)
0806 06:45:51.110890 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.110913 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:51.111063 5258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:40085: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
I20260806 06:45:51.142185 5258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } attempt: 1
I20260806 06:45:51.142568 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.142454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58498)
0806 06:45:51.142507 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.142544 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:45:51.142930 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.142839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58514)
0806 06:45:51.142883 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.142906 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:51.143018 5258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:40085: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
W20260806 06:45:51.158353 5342 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:51.158527 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:51.158542 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:51.158561 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:51.159648 5342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:51.159704 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:51.159718 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:51.159730 5342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:51.159742 5342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:51.161228 5342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:33399,127.14.51.125:41929,127.14.51.124:40085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:40085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.51.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:51.161422 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:51.161582 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:45:51.166208 5342 server_base.cc:1034] running on GCE node
W20260806 06:45:51.166203 5348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:51.166184 5350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:51.166177 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:51.166577 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:51.166791 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:51.167959 5342 hybrid_clock.cc:648] HybridClock initialized: now 1785998751167950 us; error 39 us; skew 500 ppm
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998751, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:51.170310 5342 init.cc:377] Logged in from keytab as kudu/127.14.51.124@KRBTEST.COM (short username kudu)
I20260806 06:45:51.170763 5342 webserver.cc:460] Webserver started at http://127.14.51.124:43783/ using document root <none> and password file <none>
I20260806 06:45:51.170985 5342 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:51.171027 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:51.171113 5342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:51.171458 5335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } attempt: 2
I20260806 06:45:51.171790 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.171684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58530)
0806 06:45:51.171732 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.171761 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:45:51.172106 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.172053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58536)
0806 06:45:51.172075 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.172094 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 06:45:51.172204 5335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:40085: Network error: Client connection negotiation failed: client connection to 127.14.51.124:40085: connect: Connection refused (error 111)
I20260806 06:45:51.172276 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data/instance:
uuid: "737a5847b0b940f081a107870a41dc4b"
format_stamp: "Formatted at 2026-08-06 06:45:51 on dist-test-slave-1xrh"
server_key: "808868d678552498d34d1e6b070a0e6d"
server_key_iv: "fc08991d414c0c626e10df81a3d1e5c0"
server_key_version: "encryptionkey@0"
I20260806 06:45:51.172557 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal/instance:
uuid: "737a5847b0b940f081a107870a41dc4b"
format_stamp: "Formatted at 2026-08-06 06:45:51 on dist-test-slave-1xrh"
server_key: "808868d678552498d34d1e6b070a0e6d"
server_key_iv: "fc08991d414c0c626e10df81a3d1e5c0"
server_key_version: "encryptionkey@0"
I20260806 06:45:51.173743 5342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:45:51.174552 5358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:51.174762 5342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:45:51.174839 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "737a5847b0b940f081a107870a41dc4b"
format_stamp: "Formatted at 2026-08-06 06:45:51 on dist-test-slave-1xrh"
server_key: "808868d678552498d34d1e6b070a0e6d"
server_key_iv: "fc08991d414c0c626e10df81a3d1e5c0"
server_key_version: "encryptionkey@0"
I20260806 06:45:51.174890 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:51.184726 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:51.186316 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:51.186417 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:51.190881 5342 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.124:40085
I20260806 06:45:51.190912 5414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.124:40085 every 8 connection(s)
I20260806 06:45:51.191325 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
I20260806 06:45:51.192242 5415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:51.193671 5415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998751, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:51.200866 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5342
I20260806 06:45:51.200963 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998482572191-14541-0/minicluster-data/master-2/wal/instance
I20260806 06:45:51.201251 14541 external_mini_cluster.cc:1428] Setting key aaa242fc527f0eb2f96734412d202447
I20260806 06:45:51.202569 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.194331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.14.51.126:33399)
0806 06:45:51.194388 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.194392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.194576 (+ 184us) server_negotiation.cc:407] Connection header received
0806 06:45:51.194988 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.194989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.194997 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.195060 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.195673 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.196118 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.197013 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.197186 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.197236 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.199245 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.199246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.199247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.199254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.200995 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.201484 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.201486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.201486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.201559 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.201864 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.201865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.201865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.202005 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.202083 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.202435 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.202510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:45:51.202790 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.194286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:33399 (local address 127.0.0.1:48084)
0806 06:45:51.194441 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.194492 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.194914 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.195175 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.195179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.195420 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.195602 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.195605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.196193 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.196194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.196936 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.196937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.197272 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.197273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.197367 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.197789 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.197796 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.199099 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.201094 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.201102 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.201104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.201406 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.201633 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.201633 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.201634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.201781 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.202090 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.202097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.202285 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.202404 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.202578 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260806 06:45:51.203465 5415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998751, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:51.210448 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.203756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.14.51.125:41929)
0806 06:45:51.203810 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.203812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.203821 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:51.203849 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.203850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.203858 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.203914 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.204185 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.204632 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.205313 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.205534 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.205610 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.207915 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.207916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.207917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.207924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.209372 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209676 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.209677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.209678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.209733 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209941 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.209942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.209942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.210057 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.210129 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.210284 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.210357 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:45:51.210448 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.203681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:41929 (local address 127.0.0.1:45598)
0806 06:45:51.203732 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.203743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.203784 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.203919 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.203919 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.204057 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.204097 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.204098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.204711 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.204712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.205210 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.205212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.205608 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.205608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.205670 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.206066 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.206070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.207831 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.209440 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.209441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.209603 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.209810 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.209812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.209868 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.210133 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.210134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.210174 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.210258 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.210357 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:45:51.210587 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.201679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.14.51.124:40085)
0806 06:45:51.201815 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.201821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.201864 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:45:51.201923 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.201925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.201956 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.202037 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.202798 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.203874 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.204603 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.204902 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.204990 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.207407 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.207413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.207414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.207419 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.209301 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209661 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.209664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.209665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.209729 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.209930 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.209931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.209931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.210072 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.210216 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.210296 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.210438 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 06:45:51.211310 5415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998751, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:51.218142 5258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } attempt: 2
I20260806 06:45:51.218437 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.211570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.14.51.124:40085)
0806 06:45:51.211600 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.211601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.211608 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:51.211666 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.211667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.211673 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.211725 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.211964 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.212688 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.213598 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.213905 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.213973 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.215804 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.215805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.215806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.215812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.217365 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.217686 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.217687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.217687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.217736 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.217927 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.217927 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.217928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.218027 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.218097 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.218237 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.218345 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":14}
I20260806 06:45:51.218642 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.211524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58560)
0806 06:45:51.211562 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.211571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.211603 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.211729 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.211730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.211870 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.211903 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.211904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.212759 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.212760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.213520 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.213522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.213991 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.213992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.214039 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.214483 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.214488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.215710 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.217461 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.217462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.217462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.217619 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.217808 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.217808 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.217809 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.217859 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.218090 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.218091 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.218125 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.218194 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.218285 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:45:51.218969 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.211305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.14.51.126:33399)
0806 06:45:51.211345 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.211347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.211355 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:45:51.211402 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.211403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.211412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.211467 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.212040 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.212734 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.213638 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.213914 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.213973 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.216287 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.216288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.216288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.216307 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.217548 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.217880 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.217881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.217882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.217933 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.218153 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.218154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.218155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.218296 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.218441 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.218812 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.218889 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:45:51.220166 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: Bootstrap starting.
I20260806 06:45:51.220969 5415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:51.221303 5415 log.cc:824] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:51.222033 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: No bootstrap required, opened a new log
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:45:51.223665 5415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.223820 5415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737a5847b0b940f081a107870a41dc4b, State: Initialized, Role: FOLLOWER
I20260806 06:45:51.223923 5415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.224207 5415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:51.225317 5422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.225405 5422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
W20260806 06:45:51.226015 5433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:51.226074 5433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:51.227439 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.218366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58576)
0806 06:45:51.218448 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.218459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.218506 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.218640 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.218641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.218800 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.218843 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.218845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.219870 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.219871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.220410 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.220412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.220848 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.220849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.220912 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.222056 (+ 1144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.222061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.223344 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.226363 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.226364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.226365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.226581 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.226789 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.226790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.226790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.226845 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.227163 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.227165 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.227205 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.227251 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.227334 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 06:45:51.227854 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.218437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.14.51.124:40085)
0806 06:45:51.218499 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.218501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.218509 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:45:51.218572 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.218573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.218581 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.218637 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.218920 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.219795 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.220503 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.220765 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.221131 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.223437 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.223438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.223438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.223445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.226292 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.226656 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.226657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.226657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.226719 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.226911 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.226912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.226912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.227054 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.227622 (+ 568us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.227667 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.227750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47,"thread_start_us":82,"threads_started":1}
I20260806 06:45:51.229126 5258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: Bootstrap starting.
I20260806 06:45:51.226549 5342 master_runner.cc:186] Error getting master registration for 127.14.51.126:33399: OK, instance_id { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" instance_seqno: 1785998750872627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:45:51.229331 5342 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:45:51.229806 5258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:51.229916 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.220541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45604 (local address 127.14.51.125:41929)
0806 06:45:51.220614 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.220616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.220628 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:45:51.220677 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.220678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.220689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.220754 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.221322 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.221913 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.222833 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.223151 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.223219 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.226250 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.226251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.226252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.226258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.227936 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.228752 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.228753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.228754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.228808 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.229004 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.229005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.229005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.229118 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.229275 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.229737 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.229807 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:45:51.230098 5258 log.cc:824] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:51.230897 5258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: No bootstrap required, opened a new log
I20260806 06:45:51.231158 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:51.232100 5228 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48098
I20260806 06:45:51.232664 5258 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.232777 5258 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb66ba90ddec4f93bfb462ed76a5ddac, State: Initialized, Role: FOLLOWER
I20260806 06:45:51.232851 5258 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.233119 5436 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.233197 5258 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:51.233224 5228 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:45:51.233246 5436 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.233284 5228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:51.234230 5228 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.234467 5228 leader_election.cc:288] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [CANDIDATE]: Term 1 election: Requested vote from peers f5828198a5f2404cb72ad569cb165623 (127.14.51.125:41929), 737a5847b0b940f081a107870a41dc4b (127.14.51.124:40085)
I20260806 06:45:51.234905 5388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "737a5847b0b940f081a107870a41dc4b"
I20260806 06:45:51.235010 5388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:51.236438 5388 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb66ba90ddec4f93bfb462ed76a5ddac in term 1.
I20260806 06:45:51.236678 5202 leader_election.cc:302] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 737a5847b0b940f081a107870a41dc4b, cb66ba90ddec4f93bfb462ed76a5ddac; no voters:
I20260806 06:45:51.236791 5436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:51.236956 5436 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [term 1 LEADER]: Becoming Leader. State: Replica: cb66ba90ddec4f93bfb462ed76a5ddac, State: Running, Role: LEADER
I20260806 06:45:51.237056 5436 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.237458 5436 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb66ba90ddec4f93bfb462ed76a5ddac. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.237531 5436 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:51.236110 5186 master_runner.cc:186] Error getting master registration for 127.14.51.125:41929: OK, instance_id { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" instance_seqno: 1785998751021454 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:45:51.238709 5186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:45:51.242004 5447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:51.242056 5447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:51.242168 5457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:51.242313 5457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:51.245724 5388 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [term 1 FOLLOWER]: Refusing update from remote peer cb66ba90ddec4f93bfb462ed76a5ddac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:51.246043 5452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [LEADER]: Connected to new peer: Peer: permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:51.247412 5422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb66ba90ddec4f93bfb462ed76a5ddac. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.247524 5422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.248515 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.236828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.14.51.125:41929)
0806 06:45:51.236863 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.236866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.237710 (+ 844us) server_negotiation.cc:407] Connection header received
0806 06:45:51.237796 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.237797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.237806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.237831 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.238375 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.239110 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.240181 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.240997 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.241062 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.243491 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.243492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.243492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.243499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.245129 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.245539 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.245541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.245541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.245620 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.245834 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.245835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.245836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.245963 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.246043 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.248341 (+ 2298us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.248413 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:51.249719 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.237159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.14.51.126:33399)
0806 06:45:51.237190 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.237191 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.238461 (+ 1270us) server_negotiation.cc:407] Connection header received
0806 06:45:51.238556 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.238557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.238567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.238593 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.239277 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.239988 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.240661 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.240917 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.240971 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.246248 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.246249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.246249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.246256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.247927 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.248776 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.248777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.248778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.248838 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.249076 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.249077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.249077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.249211 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.249296 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.249516 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.249608 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:51.253484 5452 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.253571 5452 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:51.253923 5422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.254014 5422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.254117 5457 catalog_manager.cc:1347] Generated new cluster ID: 589151ce822141748c2287a8037ae124
I20260806 06:45:51.254160 5457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:51.259451 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.237445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.14.51.124:40085)
0806 06:45:51.237475 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.237477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.240762 (+ 3285us) server_negotiation.cc:407] Connection header received
0806 06:45:51.241278 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.241279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.241287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.241314 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.243449 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.244178 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.247902 (+ 3724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.248191 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.248233 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.250738 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.250739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.250739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.250746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.252374 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.258432 (+ 6058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.258433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.258433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.258490 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.258851 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.258851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.258852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.258971 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.259046 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.259308 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.259374 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:45:51.263280 5335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } attempt: 3
Aug 06 06:45:51 dist-test-slave-1xrh krb5kdc[5173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998750, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:45:51.270105 5460 mvcc.cc:205] Tried to move back new op lower bound from 7315450885092491264 to 7315450885067751424. Current Snapshot: MvccSnapshot[applied={T|T < 7315450885092491264}]
I20260806 06:45:51.270983 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.263491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:40085 (local address 127.0.0.1:58598)
0806 06:45:51.263529 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:51.263540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:51.263576 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:51.263699 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:51.263700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:51.263871 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:51.263908 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.263910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.264698 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.264699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.265491 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:51.265493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.265925 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.265926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:51.265978 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.266357 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:51.266361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:51.267839 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:51.269909 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.269910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.269911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.270163 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.270387 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:51.270388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:51.270388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:51.270450 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:51.270710 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:51.270711 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:51.270758 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:45:51.270810 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:45:51.270893 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:45:51.271291 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:51.263580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.14.51.124:40085)
0806 06:45:51.263612 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.263614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.263621 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:51.263639 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.263639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.263647 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.263693 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.263963 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.264636 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.265563 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:51.265855 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:51.265919 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:51.267905 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:51.267907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:51.267907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:51.267914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:51.269846 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.270255 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.270256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.270256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.270316 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:51.270517 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:51.270517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:51.270518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:51.270632 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:51.270679 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:51.271150 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 06:45:51.271215 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:45:51.272610 5335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: Bootstrap starting.
I20260806 06:45:51.273247 5335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:51.273511 5335 log.cc:824] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:51.282040 5335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: No bootstrap required, opened a new log
I20260806 06:45:51.283834 5335 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.283964 5335 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5828198a5f2404cb72ad569cb165623, State: Initialized, Role: FOLLOWER
I20260806 06:45:51.284036 5335 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } }
I20260806 06:45:51.284297 5335 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:51.284327 5308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f5828198a5f2404cb72ad569cb165623"
I20260806 06:45:51.284443 5308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:51.285295 5308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb66ba90ddec4f93bfb462ed76a5ddac in term 1.
I20260806 06:45:51.285660 5307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [term 1 FOLLOWER]: Refusing update from remote peer cb66ba90ddec4f93bfb462ed76a5ddac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:51.286034 5436 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:51.291410 5466 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.291543 5466 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.291623 5466 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb66ba90ddec4f93bfb462ed76a5ddac. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.291712 5466 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.291770 5466 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.291847 5466 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:51.292436 5479 mvcc.cc:205] Tried to move back new op lower bound from 7315450885092491264 to 7315450885067751424. Current Snapshot: MvccSnapshot[applied={T|T < 7315450885092491264}]
I20260806 06:45:51.295609 5436 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb66ba90ddec4f93bfb462ed76a5ddac" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 33399 } } peers { permanent_uuid: "f5828198a5f2404cb72ad569cb165623" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 41929 } } peers { permanent_uuid: "737a5847b0b940f081a107870a41dc4b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 40085 } } }
I20260806 06:45:51.295707 5436 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:51.297453 5457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:51.297966 5457 catalog_manager.cc:1504] Loading token signing keys...
W20260806 06:45:51.291625 5477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:51.299283 5477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:51.302094 5457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb66ba90ddec4f93bfb462ed76a5ddac: Generated new TSK 0
I20260806 06:45:51.302592 5457 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:45:51.330448 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.329480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.14.51.125:41929)
0806 06:45:51.329523 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.329525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.329807 (+ 282us) server_negotiation.cc:407] Connection header received
0806 06:45:51.329908 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.329909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.329920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.329950 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.330385 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45616: BlockingRecv error: recv got EOF from 127.0.0.1:45616 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:45:51.332024 5263 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.331381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.14.51.126:33399)
0806 06:45:51.331422 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.331424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.331585 (+ 161us) server_negotiation.cc:407] Connection header received
0806 06:45:51.331610 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.331610 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.331620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.331660 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.331949 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48104: BlockingRecv error: recv got EOF from 127.0.0.1:48104 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:45:51.332846 5419 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.332257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.14.51.124:40085)
0806 06:45:51.332299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.332301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.332437 (+ 136us) server_negotiation.cc:407] Connection header received
0806 06:45:51.332460 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.332461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.332469 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.332512 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.332794 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58612: BlockingRecv error: recv got EOF from 127.0.0.1:58612 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:45:51.334942 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.334228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.14.51.125:41929)
0806 06:45:51.334260 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.334262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.334457 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:45:51.334559 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.334560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.334576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.334604 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.334900 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45628: BlockingRecv error: recv got EOF from 127.0.0.1:45628 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:51.337774 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.337038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45630 (local address 127.14.51.125:41929)
0806 06:45:51.337069 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.337070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.337318 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:45:51.337408 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.337408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.337417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.337445 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.337735 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45630: BlockingRecv error: recv got EOF from 127.0.0.1:45630 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:51.341449 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.340727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.14.51.125:41929)
0806 06:45:51.340759 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.340761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.341002 (+ 241us) server_negotiation.cc:407] Connection header received
0806 06:45:51.341093 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.341094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.341103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.341130 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.341410 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45642: BlockingRecv error: recv got EOF from 127.0.0.1:45642 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:51.347254 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.346445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.14.51.125:41929)
0806 06:45:51.346485 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.346487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.346716 (+ 229us) server_negotiation.cc:407] Connection header received
0806 06:45:51.346801 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.346802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.346811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.346837 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.347216 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45650: BlockingRecv error: recv got EOF from 127.0.0.1:45650 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 06:45:51.356976 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.356239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.14.51.125:41929)
0806 06:45:51.356270 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.356272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.356505 (+ 233us) server_negotiation.cc:407] Connection header received
0806 06:45:51.356598 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.356599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.356608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.356635 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.356932 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45662: BlockingRecv error: recv got EOF from 127.0.0.1:45662 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:51.374923 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.374096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.14.51.125:41929)
0806 06:45:51.374132 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.374134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.374391 (+ 257us) server_negotiation.cc:407] Connection header received
0806 06:45:51.374488 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.374489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.374501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.374531 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.374875 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45676: BlockingRecv error: recv got EOF from 127.0.0.1:45676 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:45:51.409138 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.408188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.14.51.125:41929)
0806 06:45:51.408232 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.408234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.408546 (+ 312us) server_negotiation.cc:407] Connection header received
0806 06:45:51.408649 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.408650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.408662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.408692 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.409076 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45688: BlockingRecv error: recv got EOF from 127.0.0.1:45688 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:45:51.475188 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.474445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.14.51.125:41929)
0806 06:45:51.474521 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.474523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.474638 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:45:51.474687 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.474688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.474698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.474750 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.475088 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45698: BlockingRecv error: recv got EOF from 127.0.0.1:45698 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 06:45:51.605134 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.604324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.14.51.125:41929)
0806 06:45:51.604419 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.604422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.604531 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:45:51.604632 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.604634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.604646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.604722 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.605067 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45706: BlockingRecv error: recv got EOF from 127.0.0.1:45706 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 06:45:51.862880 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:51.862232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.14.51.125:41929)
0806 06:45:51.862293 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:51.862295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:51.862310 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:45:51.862396 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:51.862398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:51.862409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:51.862460 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:51.862812 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45712: BlockingRecv error: recv got EOF from 127.0.0.1:45712 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:45:52.226462 5433 catalog_manager.cc:1259] Loaded cluster ID: 589151ce822141748c2287a8037ae124
I20260806 06:45:52.226574 5433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: loading cluster ID for follower catalog manager: success
I20260806 06:45:52.227458 5433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: acquiring CA information for follower catalog manager: success
I20260806 06:45:52.227758 5433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 737a5847b0b940f081a107870a41dc4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:45:52.299880 5477 catalog_manager.cc:1259] Loaded cluster ID: 589151ce822141748c2287a8037ae124
I20260806 06:45:52.300009 5477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: loading cluster ID for follower catalog manager: success
I20260806 06:45:52.300846 5477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: acquiring CA information for follower catalog manager: success
I20260806 06:45:52.301153 5477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5828198a5f2404cb72ad569cb165623: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:45:52.378943 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:52.376127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.14.51.125:41929)
0806 06:45:52.376256 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:52.376258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:52.376338 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:45:52.376433 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:52.376434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:52.376505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:52.376594 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:52.376750 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.377289 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.378101 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.378307 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.378389 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:52.378571 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:52.378688 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:52.378778 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:52.378817 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:45:52.378839 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 06:45:52.383010 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:52.380765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.14.51.124:40085)
0806 06:45:52.380864 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:52.380866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:52.380893 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:45:52.380958 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:52.380959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:52.380996 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:52.381062 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:52.381165 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.381621 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.382311 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.382502 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.382549 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:52.382678 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:52.382766 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:52.382818 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:52.382888 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:45:52.382903 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260806 06:45:52.387458 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:52.384764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.14.51.125:41929)
0806 06:45:52.384802 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:52.384804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:52.385033 (+ 229us) server_negotiation.cc:407] Connection header received
0806 06:45:52.385092 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:52.385094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:52.385104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:52.385157 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:52.385335 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.385872 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.386685 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.386949 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.387009 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:52.387196 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:52.387229 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:52.387279 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:52.387353 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:45:52.387373 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:45:52.388022 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:52.384768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.14.51.126:33399)
0806 06:45:52.384863 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:52.384866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:52.384875 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:52.384952 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:52.384953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:52.385060 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:52.385165 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:52.385289 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.385914 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.386987 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.387395 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.387449 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:52.387675 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:52.387773 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:52.387827 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:52.387917 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:45:52.387939 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":52,"threads_started":1}
I20260806 06:45:52.388698 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:52.384985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.14.51.124:40085)
0806 06:45:52.385044 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:52.385045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:52.385167 (+ 122us) server_negotiation.cc:407] Connection header received
0806 06:45:52.385244 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:52.385245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:52.385255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:52.385310 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:52.386097 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.386790 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.387926 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:52.388229 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:52.388287 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:52.388433 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:52.388458 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:52.388532 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:52.388593 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:45:52.388618 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:45:52.389885 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5186
I20260806 06:45:52.395998 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5266
I20260806 06:45:52.400970 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5342
2026-08-06T06:45:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2830 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:45:52 dist-test-slave-1xrh krb5kdc[5567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:45:52 dist-test-slave-1xrh krb5kdc[5567](info): set up 2 sockets
Aug 06 06:45:52 dist-test-slave-1xrh krb5kdc[5567](info): commencing operation
krb5kdc: starting...
W20260806 06:45:53.463018 14541 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/test-user.keytab.
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:45:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:45:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.51.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
Entry for principal kudu/127.14.51.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab.
I20260806 06:45:53.580006 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34371
--webserver_interface=127.14.51.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:45:53.659022 5580 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:53.659288 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:53.659320 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:53.659363 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:53.660472 5580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:53.660537 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:53.660561 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:53.660591 5580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:53.660617 5580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:53.662081 5580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.126.keytab
--principal=kudu/127.14.51.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.126:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.51.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:53.662343 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:53.662554 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:53.667351 5591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:53.667363 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:53.667451 5580 server_base.cc:1034] running on GCE node
W20260806 06:45:53.667584 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:53.667852 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:53.668154 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:53.669308 5580 hybrid_clock.cc:648] HybridClock initialized: now 1785998753669304 us; error 50 us; skew 500 ppm
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:53.671811 5580 init.cc:377] Logged in from keytab as kudu/127.14.51.126@KRBTEST.COM (short username kudu)
I20260806 06:45:53.672236 5580 webserver.cc:460] Webserver started at http://127.14.51.126:45355/ using document root <none> and password file <none>
I20260806 06:45:53.672446 5580 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:53.672518 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:53.672679 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:53.673679 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data/instance:
uuid: "c9f21ff29d884940b4c2be2e102a6228"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "e28c27af2de7311a4cf6229cfd8fba9d"
server_key_iv: "b28c24f19bea97e541570ceaedb8e8ba"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.674052 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance:
uuid: "c9f21ff29d884940b4c2be2e102a6228"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "e28c27af2de7311a4cf6229cfd8fba9d"
server_key_iv: "b28c24f19bea97e541570ceaedb8e8ba"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.675384 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:45:53.676049 5598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:53.676221 5580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:53.676321 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal
uuid: "c9f21ff29d884940b4c2be2e102a6228"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "e28c27af2de7311a4cf6229cfd8fba9d"
server_key_iv: "b28c24f19bea97e541570ceaedb8e8ba"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.676396 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:53.717777 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:53.720153 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:53.720340 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:53.724262 5580 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.126:34371
I20260806 06:45:53.724315 5652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.126:34371 every 8 connection(s)
I20260806 06:45:53.724691 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/data/info.pb
I20260806 06:45:53.725637 5653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:53.727234 5653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:53.734764 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5580
I20260806 06:45:53.734875 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-0/wal/instance
I20260806 06:45:53.735085 14541 external_mini_cluster.cc:1428] Setting key c8a60d8507cd1b3066dc08b6d7a590b7
I20260806 06:45:53.736434 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.727938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34371 (local address 127.0.0.1:54788)
0806 06:45:53.728094 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.728128 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:53.728620 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:53.728905 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:53.728907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:53.729179 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:53.729322 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.729327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.730333 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.730334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.730973 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.730974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.731541 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.731542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.731608 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.732018 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:53.732025 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:53.733319 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:53.734932 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.734959 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.734961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.735234 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.735511 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.735512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.735512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.735609 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.736025 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:53.736029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:53.736100 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 06:45:53.736168 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:45:53.736283 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260806 06:45:53.736552 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.728092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.14.51.126:34371)
0806 06:45:53.728200 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:53.728205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:53.728236 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:45:53.728789 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:53.728791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:53.728807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:53.728900 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:53.729434 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.730234 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.731079 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.731425 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.731556 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.733394 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:53.733400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:53.733400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:53.733406 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:53.734817 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.735321 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.735325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.735325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.735429 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.735673 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.735674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.735674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.735959 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:53.736041 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:53.736228 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:45:53.736369 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":81,"thread_start_us":40,"threads_started":1}
I20260806 06:45:53.737301 5653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:53.737658 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.737541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37848)
0806 06:45:53.737589 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.737631 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:45:53.737807 5600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
I20260806 06:45:53.738221 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.738108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37856)
0806 06:45:53.738167 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.738192 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 06:45:53.738440 5653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:43399: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.51.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
Entry for principal kudu/127.14.51.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab.
I20260806 06:45:53.759730 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:43399
--webserver_interface=127.14.51.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:53.787672 5653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } attempt: 1
I20260806 06:45:53.788147 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.788010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37864)
0806 06:45:53.788079 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.788118 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 06:45:53.788494 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.788391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37870)
0806 06:45:53.788434 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.788458 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:53.788579 5653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:43399: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
W20260806 06:45:53.839100 5661 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:53.839318 5661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:53.839336 5661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:53.839385 5661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:53.840482 5661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:53.840523 5661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:53.840538 5661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:53.840590 5661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:53.840608 5661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:53.842052 5661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.125.keytab
--principal=kudu/127.14.51.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.125:43399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.51.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:53.842304 5661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:53.842523 5661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:53.847218 5667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:53.847324 5661 server_base.cc:1034] running on GCE node
W20260806 06:45:53.847239 5666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:53.847218 5669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:53.847666 5661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:53.847913 5661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:53.849073 5661 hybrid_clock.cc:648] HybridClock initialized: now 1785998753849067 us; error 40 us; skew 500 ppm
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:53.851644 5661 init.cc:377] Logged in from keytab as kudu/127.14.51.125@KRBTEST.COM (short username kudu)
I20260806 06:45:53.852149 5661 webserver.cc:460] Webserver started at http://127.14.51.125:34165/ using document root <none> and password file <none>
I20260806 06:45:53.852386 5661 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:53.852456 5661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:53.852615 5661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:53.853653 5661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data/instance:
uuid: "a75f3d65da7a48a082648951674191d1"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "60d78dea5210c23ea1a10dabaf5a8706"
server_key_iv: "6972977008f3e28e985228041e0b414d"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.854039 5661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal/instance:
uuid: "a75f3d65da7a48a082648951674191d1"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "60d78dea5210c23ea1a10dabaf5a8706"
server_key_iv: "6972977008f3e28e985228041e0b414d"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.855414 5661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:45:53.856182 5678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:53.856359 5661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:53.856457 5661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal
uuid: "a75f3d65da7a48a082648951674191d1"
format_stamp: "Formatted at 2026-08-06 06:45:53 on dist-test-slave-1xrh"
server_key: "60d78dea5210c23ea1a10dabaf5a8706"
server_key_iv: "6972977008f3e28e985228041e0b414d"
server_key_version: "encryptionkey@0"
I20260806 06:45:53.856541 5661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:53.856700 5653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } attempt: 2
I20260806 06:45:53.857086 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.856960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37876)
0806 06:45:53.857021 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.857057 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:45:53.857545 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.857429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37880)
0806 06:45:53.857475 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.857517 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:45:53.857640 5653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.125:43399: Network error: Client connection negotiation failed: client connection to 127.14.51.125:43399: connect: Connection refused (error 111)
I20260806 06:45:53.871662 5661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:53.873508 5661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:53.873615 5661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:53.877653 5661 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.125:43399
I20260806 06:45:53.877704 5733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.125:43399 every 8 connection(s)
I20260806 06:45:53.878057 5661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/data/info.pb
I20260806 06:45:53.878994 5734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:53.880596 5734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:53.884037 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5661
I20260806 06:45:53.884138 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-1/wal/instance
I20260806 06:45:53.884330 14541 external_mini_cluster.cc:1428] Setting key 4afda7c0783ae8148b8b27818570ad2c
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:53.890570 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.881250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.14.51.126:34371)
0806 06:45:53.881308 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:53.881310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:53.881535 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:45:53.882010 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:53.882012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:53.882026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:53.882086 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:53.882692 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.883250 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.884502 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.884716 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.884779 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.887318 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:53.887319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:53.887320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:53.887327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:53.889086 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.889529 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.889530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.889531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.889596 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.889865 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.889867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.889867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.890026 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:53.890106 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:53.890369 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:45:53.890456 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:45:53.890607 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.881220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34371 (local address 127.0.0.1:54798)
0806 06:45:53.881404 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.881438 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:53.881902 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:53.882176 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:53.882179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:53.882413 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:53.882593 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.882597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.883356 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.883356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.884365 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.884368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.884828 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.884829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.884981 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.885449 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:53.885459 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:53.887231 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:53.889187 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.889194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.889196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.889443 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.889671 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.889672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.889672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.889785 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.890103 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:53.890109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:53.890224 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 06:45:53.890328 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:45:53.890446 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260806 06:45:53.891279 5734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.51.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.51.124@KRBTEST.COM" created.
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:53.900175 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.891540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37884)
0806 06:45:53.891589 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.891600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:53.891640 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:53.891918 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:53.891919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:53.892028 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:53.892071 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.892073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.893342 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.893343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.893805 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.893806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.894277 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.894278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.894347 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.894747 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:53.894751 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:53.896617 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:53.898916 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.898917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.898918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.899245 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.899497 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.899499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.899499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.899583 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.899851 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:53.899852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:53.899891 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:45:53.899961 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:45:53.900068 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:45:53.900269 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.891599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.14.51.125:43399)
0806 06:45:53.891760 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:53.891764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:53.891789 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:45:53.891814 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:53.891816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:53.891829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:53.891908 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:53.892147 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.893258 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.893879 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.894212 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.894265 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.896716 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:53.896723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:53.896724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:53.896732 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:53.898720 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.899369 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.899372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.899372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.899424 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.899654 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.899654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.899655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.899776 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:53.899870 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:53.899995 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:45:53.900137 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260806 06:45:53.900849 5734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:53.901227 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.901087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45604)
0806 06:45:53.901139 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.901192 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:45:53.901394 5683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
I20260806 06:45:53.901917 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.901807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45610)
0806 06:45:53.901859 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.901892 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:45:53.902067 5734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:42167: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
Entry for principal kudu/127.14.51.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab.
I20260806 06:45:53.913805 14541 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN7WBht/build/release/bin/kudu
/tmp/dist-test-taskN7WBht/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:42167
--webserver_interface=127.14.51.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.51.84:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:45:53.948791 5653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } attempt: 3
I20260806 06:45:53.951292 5734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } attempt: 1
I20260806 06:45:53.951704 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.951583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45620)
0806 06:45:53.951627 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.951667 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:45:53.952734 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.952615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45624)
0806 06:45:53.952671 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.952706 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:45:53.952848 5734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:42167: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Aug 06 06:45:53 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:53.955935 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.949234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.14.51.125:43399)
0806 06:45:53.949318 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:53.949321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:53.949333 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:45:53.949363 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:53.949364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:53.949374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:53.949440 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:53.949770 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.950258 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.950952 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.951243 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.951320 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.953545 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:53.953547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:53.953547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:53.953553 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:53.954868 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.955259 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.955260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.955260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.955311 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:53.955532 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:53.955532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:53.955532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:53.955629 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:53.955680 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:53.955829 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:45:53.955883 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:45:53.956003 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.949146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37894)
0806 06:45:53.949210 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.949228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:53.949304 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:53.949445 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:53.949446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:53.949614 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:53.949682 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.949684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.950342 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.950344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.950867 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:53.950868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:53.951317 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:53.951318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:53.951406 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:53.951946 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:53.951952 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:53.953445 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:53.954950 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.954951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.954952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.955190 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.955398 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:53.955399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:53.955399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:53.955467 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:53.955689 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:53.955690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:53.955739 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:45:53.955813 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:45:53.955909 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:45:53.956408 5653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:53.956702 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.956614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45640)
0806 06:45:53.956650 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.956679 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:45:53.956985 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.956933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45644)
0806 06:45:53.956953 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.956973 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 06:45:53.957082 5653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:42167: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
I20260806 06:45:53.988216 5653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } attempt: 1
I20260806 06:45:53.988695 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.988568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45654)
0806 06:45:53.988632 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.988669 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:45:53.989066 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:53.988990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45656)
0806 06:45:53.989028 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:53.989051 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:45:53.989176 5653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:42167: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
W20260806 06:45:53.997991 5741 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:45:53.998176 5741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:45:53.998193 5741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:45:53.998214 5741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:45:53.999333 5741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:45:53.999372 5741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:45:53.999387 5741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:45:53.999399 5741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:45:53.999411 5741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:45:54.000768 5741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.51.84:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.51.126:34371,127.14.51.125:43399,127.14.51.124:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/krb5kdc/kudu_127.14.51.124.keytab
--principal=kudu/127.14.51.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.51.124:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.51.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:45:54.000952 5741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:45:54.001111 5741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:45:54.005693 5746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:54.005718 5747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:45:54.005692 5749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:45:54.005911 5741 server_base.cc:1034] running on GCE node
I20260806 06:45:54.006122 5741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:45:54.006395 5741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:45:54.007539 5741 hybrid_clock.cc:648] HybridClock initialized: now 1785998754007506 us; error 46 us; skew 500 ppm
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:45:54.010171 5741 init.cc:377] Logged in from keytab as kudu/127.14.51.124@KRBTEST.COM (short username kudu)
I20260806 06:45:54.010632 5741 webserver.cc:460] Webserver started at http://127.14.51.124:34741/ using document root <none> and password file <none>
I20260806 06:45:54.010850 5741 fs_manager.cc:359] Metadata directory not provided
I20260806 06:45:54.010895 5741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:45:54.011059 5741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:45:54.012123 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data/instance:
uuid: "cb5a34e5e1fd47929ec4381baa1d886b"
format_stamp: "Formatted at 2026-08-06 06:45:54 on dist-test-slave-1xrh"
server_key: "35c4210616cd35b43fc73c09335ba622"
server_key_iv: "338bd1ac41f4e97357484bafc39e39f2"
server_key_version: "encryptionkey@0"
I20260806 06:45:54.012481 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal/instance:
uuid: "cb5a34e5e1fd47929ec4381baa1d886b"
format_stamp: "Formatted at 2026-08-06 06:45:54 on dist-test-slave-1xrh"
server_key: "35c4210616cd35b43fc73c09335ba622"
server_key_iv: "338bd1ac41f4e97357484bafc39e39f2"
server_key_version: "encryptionkey@0"
I20260806 06:45:54.013792 5741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:45:54.014467 5756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:45:54.014642 5741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:45:54.014741 5741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data,/tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal
uuid: "cb5a34e5e1fd47929ec4381baa1d886b"
format_stamp: "Formatted at 2026-08-06 06:45:54 on dist-test-slave-1xrh"
server_key: "35c4210616cd35b43fc73c09335ba622"
server_key_iv: "338bd1ac41f4e97357484bafc39e39f2"
server_key_version: "encryptionkey@0"
I20260806 06:45:54.014822 5741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:45:54.020978 5734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } attempt: 2
I20260806 06:45:54.021361 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.021228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45660)
0806 06:45:54.021288 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.021327 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:45:54.021708 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.021620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45674)
0806 06:45:54.021664 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.021689 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:45:54.021814 5734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.51.124:42167: Network error: Client connection negotiation failed: client connection to 127.14.51.124:42167: connect: Connection refused (error 111)
I20260806 06:45:54.035074 5741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:45:54.037319 5741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:45:54.037472 5741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:45:54.041450 5741 rpc_server.cc:307] RPC server started. Bound to: 127.14.51.124:42167
I20260806 06:45:54.041479 5810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.51.124:42167 every 8 connection(s)
I20260806 06:45:54.041798 5741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/data/info.pb
I20260806 06:45:54.042706 5811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:45:54.044189 5811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:45:54.048400 14541 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN7WBht/build/release/bin/kudu as pid 5741
I20260806 06:45:54.048489 14541 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN7WBht/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998482572191-14541-0/minicluster-data/master-2/wal/instance
I20260806 06:45:54.048704 14541 external_mini_cluster.cc:1428] Setting key 1fee0b2c3ce71f9e15ed162319718c08
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:54.053063 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.044820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.14.51.126:34371)
0806 06:45:54.044881 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.044883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.045083 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:45:54.045517 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.045518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.045527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.045583 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.046116 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.046577 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.047537 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.047712 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.047764 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.050052 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.050054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.050054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.050061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.051575 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.051966 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.051967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.051968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.052023 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.052256 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.052257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.052257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.052411 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.052489 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.052864 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.052947 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:45:54.053368 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.044774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.126:34371 (local address 127.0.0.1:54810)
0806 06:45:54.044959 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.044999 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:54.045430 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:54.045647 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:54.045651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:54.045869 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:54.046050 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.046053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.046650 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.046651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.047458 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.047460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.047796 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.047797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.047887 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.048278 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:54.048287 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:54.049949 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:54.051667 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.051672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.051674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.051879 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.052092 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.052093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.052093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.052193 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.052516 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:54.052524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:54.052694 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 06:45:54.052834 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 06:45:54.053015 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260806 06:45:54.054363 5811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:54.058497 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.049104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.14.51.124:42167)
0806 06:45:54.049447 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.049451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.049472 (+ 21us) server_negotiation.cc:407] Connection header received
0806 06:45:54.049492 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.049494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.049512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.049586 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.050230 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.051317 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.052018 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.052431 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.052493 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.055602 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.055609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.055610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.055616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.057288 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.057689 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.057692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.057693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.057751 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.057933 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.057934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.057934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.058066 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.058231 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.058321 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.058395 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":67,"threads_started":1}
I20260806 06:45:54.061941 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.054679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.14.51.125:43399)
0806 06:45:54.054738 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.054740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.054791 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:45:54.054883 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.054884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.054891 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.054935 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.055274 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.055806 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.056651 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.056962 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.057038 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.058806 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.058807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.058808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.058815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.060877 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.061224 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.061225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.061225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.061289 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.061478 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.061479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.061479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.061588 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.061634 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.061789 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.061856 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:45:54.061976 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.054600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.125:43399 (local address 127.0.0.1:37902)
0806 06:45:54.054736 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.054747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:54.054819 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:54.054941 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:54.054942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:54.055102 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:54.055198 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.055200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.055899 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.055900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.056565 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.056567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.057041 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.057042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.057092 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.057455 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:54.057459 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:54.058722 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:54.060946 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.060947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.060947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.061151 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.061356 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.061356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.061357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.061419 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.061646 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:54.061647 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:54.061686 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:45:54.061754 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:45:54.061873 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 06:45:54.062350 5811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.126@KRBTEST.COM
I20260806 06:45:54.064293 5653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } attempt: 2
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.124@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.126@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:45:54.068298 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.059293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.14.51.126:34371)
0806 06:45:54.059335 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.059337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.059347 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:54.059394 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.059395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.059405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.059466 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.060062 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.060779 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.061753 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.062379 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.062604 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.064716 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.064717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.064718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.064725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.066518 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.066912 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.066913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.066913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.066972 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.067624 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.067625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.067625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.067751 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.067928 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.068128 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.068204 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:45:54.070240 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.062563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45702)
0806 06:45:54.062652 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.062660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:54.062702 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:54.062835 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:54.062836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:54.062978 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:54.063019 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.063020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.063658 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.063659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.064339 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.064342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.064832 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.064833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.064891 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.065403 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:54.065407 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:54.066774 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:54.068740 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.068741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.068741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.069119 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.069350 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.069350 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.069351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.069422 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.069955 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:54.069956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:54.069992 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:45:54.070058 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:45:54.070142 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 06:45:54.070585 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.062611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.14.51.124:42167)
0806 06:45:54.062641 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.062643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.062691 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:45:54.062757 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.062758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.062768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.062823 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.063087 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.063565 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.064426 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.064760 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.064825 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.066856 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.066857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.066857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.066863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.068678 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.069179 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.069180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.069181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.069244 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.069774 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.069775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.069775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.069886 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.069961 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.070414 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.070482 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:45:54.072063 5811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: Bootstrap starting.
I20260806 06:45:54.072101 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.064534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45710)
0806 06:45:54.064576 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.064585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:54.064622 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:54.064883 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:54.064884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:54.065042 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:54.065082 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.065084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.065967 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.065968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.066456 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.066457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.066841 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.066842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.066897 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.067296 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:54.067299 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:54.068718 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:54.071021 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.071022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.071023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.071266 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.071464 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.071464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.071465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.071529 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.071836 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:54.071837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:54.071881 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:45:54.071933 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:45:54.072034 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:45:54.072500 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.064557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.14.51.124:42167)
0806 06:45:54.064733 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.064734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.064741 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:54.064757 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.064758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.064768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.065139 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.065170 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.065884 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.066568 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.066767 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.066842 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.068796 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.068797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.068798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.068803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.070913 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.071351 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.071352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.071352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.071414 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.071603 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.071604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.071605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.071731 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.072236 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.072291 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.072364 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260806 06:45:54.072829 5811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:54.073145 5811 log.cc:824] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: Log is configured to *not* fsync() on all Append() calls
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.51.125@KRBTEST.COM
I20260806 06:45:54.073899 5653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: Bootstrap starting.
I20260806 06:45:54.073949 5811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: No bootstrap required, opened a new log
I20260806 06:45:54.074827 5653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:54.075440 5653 log.cc:824] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:54.076122 5811 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.076279 5811 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb5a34e5e1fd47929ec4381baa1d886b, State: Initialized, Role: FOLLOWER
I20260806 06:45:54.076299 5653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: No bootstrap required, opened a new log
I20260806 06:45:54.076368 5811 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.076684 5811 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:54.076651 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.076748 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.077193 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.068938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.14.51.125:43399)
0806 06:45:54.068981 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.068983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.068991 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:45:54.069013 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.069014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.069023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.069089 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.069731 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.070408 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.071412 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.071707 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.071784 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.074364 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.074365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.074365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.074372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.075989 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.076361 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.076362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.076363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.076419 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.076689 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.076690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.076690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.076816 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.076965 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.077012 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.077081 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:45:54.078584 5653 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.078713 5653 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f21ff29d884940b4c2be2e102a6228, State: Initialized, Role: FOLLOWER
I20260806 06:45:54.078814 14541 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:45:54.078787 5653 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
W20260806 06:45:54.078853 5831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:54.079010 5831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:54.079084 5653 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:54.079229 5833 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.079368 5833 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.079756 5626 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c9f21ff29d884940b4c2be2e102a6228"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54812
I20260806 06:45:54.079861 5626 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:45:54.079902 5626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:54.080724 5626 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.080956 5626 leader_election.cc:288] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [CANDIDATE]: Term 1 election: Requested vote from peers a75f3d65da7a48a082648951674191d1 (127.14.51.125:43399), cb5a34e5e1fd47929ec4381baa1d886b (127.14.51.124:42167)
W20260806 06:45:54.081405 5844 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:54.081456 5844 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:54.079696 5741 master_runner.cc:186] Error getting master registration for 127.14.51.126:34371: OK, instance_id { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" instance_seqno: 1785998753723091 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:45:54.082091 5741 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:45:54.082332 5784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9f21ff29d884940b4c2be2e102a6228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cb5a34e5e1fd47929ec4381baa1d886b"
I20260806 06:45:54.082449 5784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:54.083319 5784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f21ff29d884940b4c2be2e102a6228 in term 1.
I20260806 06:45:54.082911 5580 master_runner.cc:186] Error getting master registration for 127.14.51.125:43399: OK, instance_id { permanent_uuid: "a75f3d65da7a48a082648951674191d1" instance_seqno: 1785998753876536 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:45:54.084681 5580 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:45:54.084852 5602 leader_election.cc:302] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9f21ff29d884940b4c2be2e102a6228, cb5a34e5e1fd47929ec4381baa1d886b; no voters:
I20260806 06:45:54.084965 5833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:45:54.085160 5833 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [term 1 LEADER]: Becoming Leader. State: Replica: c9f21ff29d884940b4c2be2e102a6228, State: Running, Role: LEADER
I20260806 06:45:54.085249 5833 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.086661 5846 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f21ff29d884940b4c2be2e102a6228. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.086755 5846 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:54.087023 5853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:45:54.087381 5853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:45:54.098551 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.087172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.14.51.124:42167)
0806 06:45:54.087206 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.087208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.088094 (+ 886us) server_negotiation.cc:407] Connection header received
0806 06:45:54.088195 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.088196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.088205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.088235 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.088995 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.089745 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.090668 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.090962 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.091021 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.094514 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.094515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.094515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.094522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.096210 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.097354 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.097355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.097356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.097414 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.097624 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.097625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.097625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.097763 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.097878 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.098356 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.098440 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:45:54.100374 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.087507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37914 (local address 127.14.51.125:43399)
0806 06:45:54.087542 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.087544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.089067 (+ 1523us) server_negotiation.cc:407] Connection header received
0806 06:45:54.089150 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.089151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.089161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.089188 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.090839 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.091521 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.092500 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.092801 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.092869 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.096069 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.096071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.096071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.096078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.097914 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.099319 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.099321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.099321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.099378 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.099610 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.099611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.099611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.099760 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.099844 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.100203 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.100288 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:45:54.102207 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.087814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.14.51.126:34371)
0806 06:45:54.087850 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.087851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.090755 (+ 2904us) server_negotiation.cc:407] Connection header received
0806 06:45:54.091626 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.091627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.091640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.091687 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.093128 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.093792 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.096579 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.096886 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.096953 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.099457 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.099458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.099458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.099464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.101023 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.101403 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.101404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.101404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.101458 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.101634 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.101636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.101636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.101748 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.101839 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.102026 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.102096 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:45:54.103204 5784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [term 1 FOLLOWER]: Refusing update from remote peer c9f21ff29d884940b4c2be2e102a6228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:54.103565 5846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:45:54.105401 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f21ff29d884940b4c2be2e102a6228. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.105480 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.106393 5833 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.106487 5833 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: This master's current role is: LEADER
I20260806 06:45:54.106884 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.106963 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.107098 5853 catalog_manager.cc:1347] Generated new cluster ID: 4b03640c186f48cea7cfb185dfe2c352
I20260806 06:45:54.107154 5853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:45:54.107220 5857 mvcc.cc:205] Tried to move back new op lower bound from 7315450896800002048 to 7315450896734019584. Current Snapshot: MvccSnapshot[applied={T|T < 7315450896800002048 or (T in {7315450896800002048})}]
I20260806 06:45:54.112957 5734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } attempt: 3
I20260806 06:45:54.113838 5859 mvcc.cc:205] Tried to move back new op lower bound from 7315450896800002048 to 7315450896734019584. Current Snapshot: MvccSnapshot[applied={T|T < 7315450896800002048}]
Aug 06 06:45:54 dist-test-slave-1xrh krb5kdc[5567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.51.125@KRBTEST.COM for kudu/127.14.51.124@KRBTEST.COM
I20260806 06:45:54.117810 5853 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:45:54.118844 5853 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:45:54.121037 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.113201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.51.124:42167 (local address 127.0.0.1:45726)
0806 06:45:54.113248 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:45:54.113262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:45:54.113313 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:45:54.113528 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:45:54.113529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:45:54.113691 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:45:54.113734 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.113735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.114605 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.114606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.115396 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:45:54.115398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.115854 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.115855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:45:54.115921 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.116408 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:45:54.116414 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:45:54.118100 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:45:54.119999 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.120000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.120000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.120193 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.120421 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:45:54.120422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:45:54.120422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:45:54.120492 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:45:54.120764 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:45:54.120764 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:45:54.120813 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:45:54.120864 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:45:54.120943 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:45:54.121490 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:54.113340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.14.51.124:42167)
0806 06:45:54.113378 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.113380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.113389 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:45:54.113408 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.113408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.113417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.113481 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.113801 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.114534 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.115486 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:54.115781 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:54.115842 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:54.118181 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:45:54.118182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:45:54.118183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:45:54.118189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:45:54.119903 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.120271 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.120272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.120273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.120348 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:45:54.120562 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:45:54.120563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:45:54.120563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:45:54.120681 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:45:54.120730 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:54.121336 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 06:45:54.121406 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:45:54.123167 5734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: Bootstrap starting.
I20260806 06:45:54.123952 5734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: Neither blocks nor log segments found. Creating new log.
I20260806 06:45:54.124300 5734 log.cc:824] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:45:54.125066 5853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228: Generated new TSK 0
I20260806 06:45:54.125437 5853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:45:54.125792 5734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: No bootstrap required, opened a new log
I20260806 06:45:54.127832 5734 raft_consensus.cc:348] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.127964 5734 raft_consensus.cc:729] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75f3d65da7a48a082648951674191d1, State: Initialized, Role: FOLLOWER
I20260806 06:45:54.128054 5734 consensus_queue.cc:260] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } }
I20260806 06:45:54.128379 5734 sys_catalog.cc:564] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:45:54.128405 5708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9f21ff29d884940b4c2be2e102a6228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a75f3d65da7a48a082648951674191d1"
I20260806 06:45:54.128513 5708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:45:54.129412 5708 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f21ff29d884940b4c2be2e102a6228 in term 1.
I20260806 06:45:54.129710 5707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [term 1 FOLLOWER]: Refusing update from remote peer c9f21ff29d884940b4c2be2e102a6228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:45:54.129962 5846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:45:54.134303 5878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:45:54.134361 5878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:45:54.134420 5866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f21ff29d884940b4c2be2e102a6228. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.134549 5866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.134608 5866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.134656 5866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.138159 5879 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.138243 5879 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:45:54.138734 5880 mvcc.cc:205] Tried to move back new op lower bound from 7315450896888057856 to 7315450896734019584. Current Snapshot: MvccSnapshot[applied={T|T < 7315450896800002048}]
W20260806 06:45:54.157963 5816 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.156407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.14.51.124:42167)
0806 06:45:54.156447 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.156449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.157346 (+ 897us) server_negotiation.cc:407] Connection header received
0806 06:45:54.157444 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.157445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.157455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.157484 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.157901 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45730: BlockingRecv error: recv got EOF from 127.0.0.1:45730 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:45:54.158659 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.156748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.14.51.125:43399)
0806 06:45:54.156791 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.156793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.158192 (+ 1399us) server_negotiation.cc:407] Connection header received
0806 06:45:54.158283 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.158284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.158293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.158322 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.158608 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37924: BlockingRecv error: recv got EOF from 127.0.0.1:37924 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 06:45:54.159428 5656 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.157083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.14.51.126:34371)
0806 06:45:54.157120 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.157122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.158913 (+ 1791us) server_negotiation.cc:407] Connection header received
0806 06:45:54.159004 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.159005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.159015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.159043 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.159352 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54838: BlockingRecv error: recv got EOF from 127.0.0.1:54838 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:45:54.161746 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.161017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.14.51.125:43399)
0806 06:45:54.161049 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.161051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.161268 (+ 217us) server_negotiation.cc:407] Connection header received
0806 06:45:54.161366 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.161367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.161385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.161412 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.161707 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37930: BlockingRecv error: recv got EOF from 127.0.0.1:37930 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:54.164647 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.163955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.14.51.125:43399)
0806 06:45:54.163985 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.163986 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.164191 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:45:54.164279 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.164279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.164290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.164317 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.164609 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37946: BlockingRecv error: recv got EOF from 127.0.0.1:37946 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:45:54.168409 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.167713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.14.51.125:43399)
0806 06:45:54.167742 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.167744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.167962 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:45:54.168051 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.168051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.168060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.168087 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.168371 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37960: BlockingRecv error: recv got EOF from 127.0.0.1:37960 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:45:54.174134 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.173417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.14.51.125:43399)
0806 06:45:54.173446 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.173448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.173661 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:45:54.173762 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.173763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.173772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.173799 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.174094 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37974: BlockingRecv error: recv got EOF from 127.0.0.1:37974 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:45:54.187896 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.183235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.14.51.125:43399)
0806 06:45:54.183267 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.183269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.187334 (+ 4065us) server_negotiation.cc:407] Connection header received
0806 06:45:54.187436 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.187437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.187447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.187478 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.187842 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37990: BlockingRecv error: recv got EOF from 127.0.0.1:37990 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:45:54.205874 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.205033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.14.51.125:43399)
0806 06:45:54.205067 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.205069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.205317 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:45:54.205418 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.205419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.205429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.205459 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.205824 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37994: BlockingRecv error: recv got EOF from 127.0.0.1:37994 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:45:54.240255 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.239372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.14.51.125:43399)
0806 06:45:54.239417 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.239419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.239666 (+ 247us) server_negotiation.cc:407] Connection header received
0806 06:45:54.239771 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.239772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.239784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.239814 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.240194 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38010: BlockingRecv error: recv got EOF from 127.0.0.1:38010 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:45:54.306419 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.305731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.14.51.125:43399)
0806 06:45:54.305802 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.305804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.305901 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:45:54.305970 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.305971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.305981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.306025 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.306338 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38026: BlockingRecv error: recv got EOF from 127.0.0.1:38026 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 06:45:54.436527 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.435692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.14.51.125:43399)
0806 06:45:54.435776 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.435778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.435891 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:45:54.436002 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.436003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.436026 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.436089 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.436457 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38036: BlockingRecv error: recv got EOF from 127.0.0.1:38036 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:45:54.517920 5846 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9f21ff29d884940b4c2be2e102a6228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9f21ff29d884940b4c2be2e102a6228" member_type: VOTER last_known_addr { host: "127.14.51.126" port: 34371 } } peers { permanent_uuid: "a75f3d65da7a48a082648951674191d1" member_type: VOTER last_known_addr { host: "127.14.51.125" port: 43399 } } peers { permanent_uuid: "cb5a34e5e1fd47929ec4381baa1d886b" member_type: VOTER last_known_addr { host: "127.14.51.124" port: 42167 } } }
I20260806 06:45:54.518087 5846 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f21ff29d884940b4c2be2e102a6228 [sys.catalog]: This master's current role is: LEADER
W20260806 06:45:54.694557 5738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:45:54.693788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.14.51.125:43399)
0806 06:45:54.693860 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:54.693863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:54.693874 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:45:54.693982 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:54.693983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:54.693993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:54.694054 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:45:54.694490 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38048: BlockingRecv error: recv got EOF from 127.0.0.1:38048 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:45:55.079494 5831 catalog_manager.cc:1259] Loaded cluster ID: 4b03640c186f48cea7cfb185dfe2c352
I20260806 06:45:55.079559 5831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: loading cluster ID for follower catalog manager: success
I20260806 06:45:55.080461 5831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: acquiring CA information for follower catalog manager: success
I20260806 06:45:55.080833 5831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cb5a34e5e1fd47929ec4381baa1d886b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:45:55.134785 5878 catalog_manager.cc:1259] Loaded cluster ID: 4b03640c186f48cea7cfb185dfe2c352
I20260806 06:45:55.134859 5878 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: loading cluster ID for follower catalog manager: success
I20260806 06:45:55.135707 5878 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: acquiring CA information for follower catalog manager: success
I20260806 06:45:55.136020 5878 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a75f3d65da7a48a082648951674191d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:45:55.210325 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:55.207700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.14.51.125:43399)
0806 06:45:55.207812 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:55.207813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:55.207823 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:45:55.207898 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:55.207899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:55.207947 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:55.208010 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:55.208164 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.208655 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.209375 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.209568 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.209624 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:55.209763 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:55.210050 (+ 287us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:55.210132 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:55.210169 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:45:55.210189 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260806 06:45:55.214419 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:55.212005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.14.51.124:42167)
0806 06:45:55.212145 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:55.212147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:55.212154 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:45:55.212199 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:55.212200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:55.212233 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:55.212309 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:55.212414 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.212970 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.213666 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.213870 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.213962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:55.214075 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:55.214170 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:55.214252 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:55.214299 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:45:55.214336 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260806 06:45:55.219301 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:55.216010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.14.51.126:34371)
0806 06:45:55.216124 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:55.216126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:55.216336 (+ 210us) server_negotiation.cc:407] Connection header received
0806 06:45:55.216383 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:55.216384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:55.216435 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:55.216537 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:55.216640 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.217422 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.218472 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.218680 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.218757 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:55.218923 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:55.219044 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:55.219116 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:55.219193 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:45:55.219217 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260806 06:45:55.219367 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:55.216063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.14.51.124:42167)
0806 06:45:55.216103 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:55.216105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:55.216292 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:45:55.216358 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:55.216359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:55.216369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:55.216394 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:55.216561 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.217318 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.218406 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.218688 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.218754 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:55.218929 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:55.218963 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:55.219024 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:55.219272 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 06:45:55.219293 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:45:55.219769 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:45:55.216957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.14.51.125:43399)
0806 06:45:55.216994 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:45:55.216996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:45:55.217127 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:45:55.217204 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:45:55.217205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:45:55.217216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:45:55.217270 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:45:55.217414 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.217888 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.218801 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:45:55.219123 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:45:55.219182 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:45:55.219486 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:45:55.219517 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:45:55.219572 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:45:55.219678 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:45:55.219697 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:45:55.226392 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5580
I20260806 06:45:55.233119 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5661
I20260806 06:45:55.238377 14541 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN7WBht/build/release/bin/kudu with pid 5741
2026-08-06T06:45:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2837 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5668 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (272547 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST